Software Craft, Testing

Testability vs. Wtf’s per minute

Leave a comment

Lately two postings on my feed reader popped up regarding testability. While reading through Michael Boltons Testability entry, I noticed that his list is a very good one regarding testability. The problem with testability from my perspective is the little attendance it seems to get. Over the last week I was inspecting some legacy code. Legacy is meant here in the sense that Michael Feather’s pointed it out in his book “Working effectively with Legacy Code”: Code without tests. Today I did a code review and was upset about the classes I had to inspect. After even five classes I was completely upset and gave up. In the design of the classes I saw large to huge methods, dealing with each other, moving around instances of classes, where no clear repsonsibility was assigned to, variables in places, where one wouldn’t look for them, etc. While I am currently reading through Clean Code from the ObjectMentors, this makes me really upset. Not only after even ten years of test-driven development there is a lack of understanding about this practice, also there is a lack of understanding about testability. What worth is a class, that talks to three hard-coded classes during construction time? How can one get this beast under test? Dependency Injection techniques, Design Principles and all the like were completely absent on these classes. Clearly, this code is not testable – at least to 80% regarding the code coverage analysis I ran after I was able to add some basic unittests, where I could. Code lacking testability often also lacks some other problems. This is where Michael Bolton, James Bach and Bret Pettichord will turn in heuristics and checklists, the refactoring world named these as Smells.

On the Google Testing blog was an entry regarding a common problem, I also ran into several times: Why are we embarrassed to admit that we don’t know how to write tests? Based on my experience project managers and developers think that testers know the answers for all the problems hiding in the software. We get asked, “Can you test this?”, “Until when are you going to be finished?” without a clear understanding of what “tested” means or any insight what we do most of the time. “Perfect Software – and other illusions about testing” is a book from Jerry Weinberg from last year, which I still need to read through in order to know if it’s the right book to spread at my company – but I think so. If a develoepr doesn’t know about the latest or oldest or most spread technology, it’s not a problem at all. If a tester does not know how to “test” this piece of code, it is. He’s blocking the project, making it impossible to deliver on schedule – escalation! What Misko points out in his blog entry is, that the real problem behind this is also testability:

Everyone is in search of some magic test framework, technology, the know-how, which will solve the testing woes. Well I have news for you: there is no such thing. The secret in tests is in writing testable code, not in knowing some magic on testing side. And it certainly is not in some company which will sell you some test automation framework. Let me make this super clear: The secret in testing is in writing testable-code! You need to go after your developers not your test-organization.

I’d like to print this out and hang it all over the place at work.

Agile Testing, Testing

Overview of Agile Testing

Leave a comment

In the just released July issue of the Software Test and Performance Magazine there is an article from Matt Heusser, my mentor in the Miagi-Do School of Software testing, and Chris McMahon introducing to the most basic terms surrounding Agile Testing. Before they both wrote them down, I was able to provide some feedback on it. It seems to be enough feedback to get a mentioning at the end of the article. Basically I’m pleased to be able to provide my help.

Since I mentioned the Miagi-Do School, I have to make clear that the term school is not meant in the Kuhnian way. James Bach pointed out to me that he uses the term school regarding the five schools of software testing (Analytic, Standard, Quality, Context-driven and Agile) as a school of thought in such a way. This means that to adapt a mindset. Like James put me out, there are few circumstances where being driven by the context is contra-productive and made me think about three such situations. By the way: Can you think of three situations where being driven by the context of the situation is unnecessary?

From my understanding that I got from Matt’s Miagi-Do school it is not to be understood in the Kuhnian way.

Testing

Just because you can, doesn’t mean you should

Leave a comment

The following I got from a post on design principles related to object oriented code. You can find the whole enchilada here.

Today I was surprised that – while the principle of single responsibility is rather new in the software world – this principle is known for a long, long time in the testing world. Why?

On the project I’m currently working on we got our requirements as an database dump. Our developers decided to generate the configuration – our system under test – using some scripts, extracting it from the database directly and transforming it into the configuration for the software that we will deliver.

The test team that was asked to do the test automation for this configuration then was asked to built some automated tests for use in FitNesse. What they now did was to do pretty much the same as our development and have the test cases generated from the database. This may sound like a reasonable approach.

So, where’s the problem? The problem is that the resulting tests are not aware of the context. What is the context? The context is, that we need to deliver some rating software to a customer in Brasil. The tax system in Brasil is divided by the 27 states. We have four major bunches of baselines, each with about 8 to 10 variations in the baseline price (20, 40, …). The resulting configuration consists of about 4 times 10 times 27 (= 1080) different single variations. For each single variation there are about 20 to 100 single test cases necessary. The generated test suite now consists of a major bunch of tests (> 100000), which run on an overall system level against our delivered system with a test execution time of about 45 seconds. This results in an exhaustive regression test suite which takes 24 hours to execute. Due to the payload generated the FitNesse system is not handling these tests in a stable way, causing some crashes due to the amount of test cases in the overall structure.

So, restating the situation from a different perspective: The configuration gets generated and adaptation to changing requirements just takes a few hours for the developers. Test case execution is near to be impossible. On the other hand there are high test maintenance costs attached to the tests generated resulting in a high amount of rework necessary to keep up with the development team. Delivering feedback quickly to the developers is unlikely to happen. So, what’s the reason to have this high amount of never executed test cases in first place?

This is where I get to the topic of this post: Just because you can, doesn’t mean you should. Just because it is possible to generate an exhaustive test suite for test automation, does not mean you should do this. When the resulting test execution times perform badly and even cause the test system to crash once in a while due to the high amount of tests included, you should refuse to do so.

So what should I do? Clarify your mission for the testing activities. What is the goal of your testing? Do you want to find risky problems quickyl? Then find another approach. Do you want to prove that everything is fine ignoring the approach of the development team? Then you should build an exhaustive test suite – probably. (I would suggest to look for a better approach to get feedback in 90 minutes maximum on it.) Know your context. How is the configuration generated? If errors reported by your test suite will be the same for all the different states, then you should re-think your approach. If there is a complex logic attached to generating the configuration, then probably you should consider testing that logic and just send out some tracer bulletts from end to end. What you can always do is to reconsider your current approach and check for opportunities to improve. The time that you might be able to win due to this, can be taken for follow-up testing and catching up in other activities. Maybe you can spend some more time pairing with your developers on the unit tests?

Leadership

Team and coaching challenge

Leave a comment

During the last view weeks I was able to come up with a challenge related to teams and coaching. On Thursday I decided to put this challenge up on my blog after having discussed some possibilities with James Bach, Matt Heusser and Michael Bolton. Personally I would like to try to apply Weinberg‘s Rule of Three Interpretations:

If I can’t think of at least three different interpretations of what I received, I haven’t thought enough about what it might mean.
(Quality Software Management – Vol. 2 First-Order Measurement, p. 90)

Prequel

Some weeks ago James Bach challenged me on a conversation about three example situations where being aware of the context at hand is unneccessary. The answers boiled down in general to situations in which the responsibility for the contextual decisions is taken by someone else like a swimming trainer takes the responsibility for the context of her students. One week later I was involved in a team, where I was wondering if the team members were being made context-unaware over a larger timeframe.

Introduction

The team is working for a company which has made some changes to their development processes in the past two years. Beforehand each project team had a project manager dealing with the schedule and all the team issues, while software development and software testing were tracked separately most of the time each having their own lead in the project. During one larger project for an end-customer the problems piled up thus far, that the whole development team got stuck – really stuck. Everyone seemed to be involved into that project for one reason or another. Due to the high amount of escalation there was always a “topic of the day”, the most hurting issue, and it was rather hard to focus.

Some months later it was decided to organize the projects in a different way. Similar to a Scrum of Scrums teams were sub-divided into more logical teams with one team above them to steer the project towards its goal. Among the teams there are roles defined like Project Manager, Technical Lead, Architect or Quality Lead. The structure proofed to be better during the next few projects. Therefore a most recent project choosed that structure, too.

The Team

The team had to steer a project over four countries within two different timezones with a maximum of seven hours time difference. The customer gives feedback in a language that just a few of the team members understand and is not able to participate in requirements. The team that steers the project is spread around the globe. Team meetings are taking place over Skype solely due to technical problems at one location. During these meetings one person speaks about 80 percent of the time, rephrasing most of the remaining people. Most of the decisions are kept unaddressed during the meetings. There is little to no documentation, little to no planning, but the person speaking 80 percent of the time seems to have everything in his head. After half a year there is the first delivery to be made. One month before this date there seems to be little to no testing done on critical components of the software. The quality responsible person is a rather silent one, just speaking one to two sentences per meeting. Due to the separation of development and testing, the developers do not feel responsible to support the testing. Along the project there were just a few testers involved overall. The developer:tester ratio was about 2:1.

The Challenge

Given the informations from above, which particular situation would you observate to gather more informations? Suppose you got into that project, which three things would you change immediately? How? What would you tell your boss about that project? What would you tell members of that team? Giving the history of the company and the history of the project, which suggestions would you make? Feel free to provide your answers in the comments or drop me a line via mail. (Maybe I’ll come up with some follow-up questions.)

Journal, Software Craft

Take your time for improvement

1 Comment

Dave Hoover just raised an interesting point on personal improvement: Spend 50% of your working time on personal improvement. Basically I found myself in that blog post. Since about two years – since I have been asked for taking a leadership responsibility – I am constantly reading books in my spare time. Over the last year I started to read several mailing lists and blogs. I am thankful to have married a wife over the last year that is patient with me (I wonder how long she’ll stay that way).

What shruggs myself is the fact that Dave states it takes about five years to see the return on that investment. Five years seems to be a very long time. Reflecting over my life so far I must say, that I have spent more than five years already on improving personally. At the age of 15 I choose to lead a youth group at our local sports club. In my spare time I gave swimming lessons over the past thirteen years and also grew into the leader role at my sports club over the past 15 years. In the meantime – while making my diploma at the university – I was involved in three clubs at a time. Through all these I made great experiences in leadership and organisation. Over ten years I participated and helped to organise helpers for our season openings of our local open air bath. This was a great time and helped myself improve for my current job.

On the other hand I even worked for money at a local store during my university years. There I had to organise the order process for soft drinks, juice and later even alcoholic drinks. I had to organise according to market and seasonal demands and manage the surpluses. In the end there was a major rebuild of the store where I was highly engaged in. Over the years I had learned how to work together with my colleagues and contribute to critical work that is daily business at a local store.

All these experiences were a major part of the preparation for my current position. In the past I have been working with people, for people, sometimes even against people if I was convinced from the opposite. Taking some spare time now to support my journey towards mastery is just a little duty for me. Personally I just took out my calculator and tried to calculate the time that I might have left for my family with Dave’s proposal: One week has 168 hours, 6 hours per sleep each night, two hours for getting to work and back, two hours for eating, 40 hours of work, 20 hours for personal improvement leaves you with 42 hours of time you may spent with your family and for leisure activities like sports. This seems ok to me, but don’t blame me for my naive assessment of the situation. (My wife is working in retail sail and usually does not come home until 9pm due to this, so don’t try to project my situation onto you.) Take Dave’s point into account to at least try it out for a while. Personally I consider myself more of the former group of people in his statement:

I think it would be silly to try to force yourself to do it, you’d end up burning out. Really, I’m talking to two groups of people. To those who are already spending their spare time on becoming a better developer, I want you to know that your efforts will pay off, but understand that it will take between 5-10 years to see the most significant benefits of your efforts. To those who want to become a great developer but hold themselves back out of fear of failure or hard work, I hope to inspire you to give it a shot.

and Dave made me feel good about it.

Journal, Leadership, Software Craft

Grading, evaluation and good code applied to apprenticeships

Leave a comment

Today I had the opportunity to talk about some of the skills that I needed to learn while becoming a leader. The girlfriend of a brother-in-law is a teacher for primary school. Today I got into a discussion with her about grades in school. During the discussion I thought on where do teachers get to know how to grade a student. Since I had one in front of me, I took the opportunity to ask directly. She stated that there was no education or course during the time she spent on university on this. In Germany there are two years of traineeship for new teachers mandatory. She stated that during that time she learned just a few regarding the grading process. So most of the grading is done on intuition. Interestingly her description reminded myself of the very first few months of my group leader time, during which I had to do a performance review for one of my colleagues. No one had taught me up to that point how to do it, I just had one review so far for myself. There was no course back in university where I could have learned it. During my last ten to fifteen years I have been a swimming trainer and lead the swimming department at our local sports club for some time, but clearly this was the only opportunity where had experience from leading people – and these situations are less context-driven due to its nature of a teacher to a student. Even today it is still hard for me to reflect whether or not I take the proper intuition into account. (The bad news is that most of your colleagues will not take the courage to inform you of your bad decisions.)

Right before starting to write about this article, I noticed that there is a similar case for good versus bad code. What did I learn at the university about code? What had I learned about bad code? Good code? Most of the insights I have today come from my experiences at work. When I started about three years ago we had an undesigned test automation framework. There were lots of script functions without intent. Whenever I needed to put in something new, I had to go out look for the right place to do so. Additionally I was never fully aware if there already was a similar function that I should have been extended, decomposed, refactored to meet my new requirement. This was a mess. Over the past year I was able to get rid of this grown framework and introduce and grow a new one.

We focussed on good aspects of code there. Using test-driven development, refactoring and regular reflection while evolving our classes. This was a good practice and we even tried to avoid most of the deficiencies from our previous experiences. Over the last year we had some contributions from a team that did not follow your initial approach. The result is that there are currently two ways to do things. When inspecting the code I can now see the difference between good code I wrote, bad code I wrote and very bad code from that I permitted to be included into the code base. (There was an up in the findbugs remakrds, a down in the code coverage and an up in the crappyness trend after the commit operation. The bad code was visible.) The main problem is that my colleagues do not seem to bother, where the broken window rule comes in. There is much work left to do and I know that I will have to take the responsibility to clean up this mess.

The point that is striking for me is, that there is no education on these things that matter most during day-to-day work. You do not get taught the stuff to grade your students. You get able to do so by getting experiences, intuition and your own guts and hunches. Similarly there is no education for a good or a not so good worker. You have to trust yourself here. The same goes for the cleanliness of code. Despite the books I could mention (the one with just that name), there is no preparation in university for clean code. Similarly there is little to no education on good tests or not so good tests. From my perspective a major challenge for craftsmanship – be it software craftsmanship or even testing craftsmanship – is to teach these difference in an apprenticeship. This is our profession and our job to do in the years to come.

Software Craft, Testing

Testing in the days of Software Craftsmanship

1 Comment

Matt Heusser came up with the idea of the Boutique Tester. While reading through his article, I was wondering about the same striking question that arises in me since the early days of the Software Craftsmanship movement: Where are testers going to be in the Craftsman world?

Some months ago I started a series which I called Testing focus of Software Craftsmanship, covering Values, Principles I and Principles II and in addition I have planned an entry on Principles III, that I did not conclude so far. Most of the stuff in it I took from Elisabeth Hendrickson, Lisa Crispin, Janet Gregory, Brian Marick, James Bach, etc., etc. In this series I tried to raise an understanding of the testers role in the craft world while just restating the lessons most Agile teams already learned. The reason that I did not yet conclude this series is that I came to the realization that there seems to be no clear understanding of the sticky minded in the craftsman worldview for me.

Reflecting back on the lessons I learned from Agile methodologies like extreme Programming, Scrum, Crystal or Lean I remember myself being a bit jealous. Programmers were taught to use test-driven development, pair programming, big visible charts, planning poker. For me they seemed to have most of the fun. The tester simply does testing – the stuff they always do. As long as they don’t get into the way, this is wonderful… It was striking first, but when I realized that there are definitions of Exploratory Testing, Pair Testing, Retrospectives, etc., I got aware that testers also have fun activities.

A while back I proposed to rename the Agile testing school to the example-driven one. After getting some feedback from peers, I realized that there seem to be no room for school-mindsets in the Agile world and care for naming in the other schools. My initial motivation to rename the Agile school was driven by the realization that the Agile view on testing seem to be a good one for craftsmanship either. While my reasoning was based on poor assumptions, I started to realize a bigger argument behind this.

The bigger argument behind all three previously described situations is from my point of view that Software Testers have defined their craft long, long ago. Jerry Weinberg wrote in December about the way unit testing was done in the older days. Over the years there have been several thought-leaders appeared for Software testing. (I refuse to name a few here, since I know I’m too few in business to give honor the right ones.) Over and over the craft of software testing has been discussed, defined and found anew as the latest approaches to Agile testing teaches us. Just a few months back there was a heavy discussion on why Agile testers just define the terms that context-driven testers already teach since decades.

What is the difference between Software Craftspersons and Testing Craftspersons then? Testers know their craft from decades of experiencing and definition. There are more than enough books out there that teach good software testing just as not so good techniques. Software testers have most of their set of tools together to start leading their craft. For the development world this is not as true. Over the decades programming languages have come up and fallen down. Programming techniques like assembler, structured programming languages, logical programming languages, object-oriented programming languages, functional languages. The craft of software development seems to be more fluent to me as the craft of software testing. You can nearly apply all the testing techniques found in the books to your software, even to real world products as an Easy Button.

My point is that Matt raises a good point in his Boutique Tester article. Testers can directly go out and provide their testing services, gaining experiences over the years in order to improve. Maybe they can also take apprentices to teach the lessons they learned over the years. Basically I assume his proposal could work, though I’m concerned whether I would like to view myself more as a tester-developer or more as a developer-tester. In the end I think what matters most, is that there is only us and testers and developers form some kind of a symbiosis, just as automated and manual testing does. It’s a good thing that we’re not all generalists, but we should keep our mind open for different views of the problem at hand to compose an improved solution to it.

Leadership, Retrospectives

Facets of an Agile Team

Leave a comment

Brian Marick started a series of blog posts on the seven pillars of an Agile team. Brian, Chet Hendrickson, Ron Jeffries, Robert Martin and James Shore came up with seven basic aspects of an Agile team:

  • Product sense
  • Collaboration
  • Focus on Business Value
  • Supportive Culture
  • Confidence
  • Technical Excellence
  • Self Improvement

Brian already started a first description on Product sense with more to come.

His description on the spider graph approach used during retrospectives reminded myself on the A Better Team questionaire based on James Shore’s and Shane Warden’s book The Art Of Agile Development. Since James contributed to the seven items, I believe this is no coincidence.

Software Craft, Testing

Three stone cutters

Leave a comment

Ivan Sanchez put up a blog entry on a parable: Three stone cutters. He challenged my thoughts with the following question:

Sometimes I have this feeling that we as professionals are frequently trying to be more like the [“I’m a great stone cutter. I can use all my techniques to produce the best shaped stone“] stone cutter than the [“I’m a stone cutter and I’m building a cathedral”] one. Am I the only one?

Due to the analgoy I felt the urge to reply to this question with Elisabeth Hendricksons Lost in Translation and Gojko Adzics Bridging the Communication Gap. Most of the stone cutters I meet in daily business think they are creating a cathedral, while building a tower of babel. The biggest obstacle to efficient project success lies from my point of view in communication among team members.

This principle was discovered in his research on successful project in the 90s and early 2000s by Alistair Cockburn, see for example Methodology Space or Software development as a Cooperative Game (Warning: don’t get lost on his wiki site just as I did several times). Software Development is done for people by people. Since people are strong on communicating and looking around, software development as whole should strengthen this particular facet of us. A cathedrale is of no use if the entrance is on the top of the highest tower. Though during day to day work I see software developers lacking early feedbacks, that do not see the problem.

The industry I’m working in has a high changing rate of requirements. In the mobile phone industry there really are impacts from competitors, that you will have to react on to stay in business. For me this means to reduce the time to market for the product we’re building as massive as reasonable from a quality perspective. This also means to communicate with our customers spread all over the world in different time zones – even in distributed teams. This means that I need to find technical and motivational ways to bring people together that do not see each other on a day to day basis in the office. Just last week I we performed a Lessons Learned workshop on one of our products, where our colleagues in Malaysia took there web cam the first time and showed us how the world was looking outside their office. (It was during a refreshening break, so don’t get wild on wasted resources, my dear boss.) This lead to personal touch and a lower personal distance between the two teams – at least during that session.

Anyways I wonder why these moments happen too little. Instead I see teams – erm basically groups of people – working without communicating with each other though they need some information from the co-worker directly near to them and wondering in the end why there is an integration problem. Another point is the problem of open interfaces (lacking communication of protocols and the like), open scope (lacking the communication of content of the project), open everything. Clearly, for me it seems there is no way to communicate too much, it’s more the opposite that I bother about.

Software Testing, Craft, Leadership and beyond