In the dialog about my recent blog entry about the Definition of Done, I was able to refine my mental model about “Done”. Here is a write-up of it.
Continue reading From the Definition of Done to “Are we done, yet?”Category Archives: Test Driven Development
Agile Testing Days Berlin III – First presentations
Agile Testing Days Website
On the Agile Testing Days on Tuesday the first presentations settled off. In the meantime pictures from the conference have been put up. Here are portions of my notes from Tuesday.
Agile Testing Days Berlin II – Monday evening dinner conversations
Agile Testing Days Website
On Monday evening of the Agile Testing Days all speakers were invited to go to lunch from the organizators. I had the dear honor to sit next to Tom Gilb with Mary & Tom Poppendieck directly facing. Though I was not able to follow the complete conversation in-depth, I was able to get most of their points. Unfortunately I didn’t take notes on everything discussed there, but I denoted two things on the next day from that conversation.
Agile Practices in a Traditional Environment
Agile Practices in a Traditional Environment from Markus Gärtner
I put up my slides to my presentation at the Agile Testing Days. It was a very tough talk for me. The first conference presentation I made. Before the presentation I was very nervous, but I had a good feeling afterwards. In addition I realized that I will have to work on my entertaining skills for future presentations. Among the things that I just verbally mentioned during the talk – there was no video taken from the talk – are three more successors of our work. The first one is the outcome that over the course of our work a colleague transitioned from the testing group to the development group as one outcome. Another thing that I mentioned is the fact that I was able to learn Java programming sufficiently enough to contribute to the test framework FitNesse over the course of this year. Third recently I paired with a developer on fixing a bug in the production code. I noted the bug when it was first filed, wrote a failing acceptance test for it, and decided to help the developer with the fix, since I would have been blocked otherwise. I showed him how to rewrite the rather complex if-then-else chain the code showed up with – not covered by fast-feedback unittests – and afterwards we fixed the bug and delivered the fix.
Since I know that it will be hard to understand anything from my rather condensed presentation style format, I also decided to put up the nine pages of paper I wrote. You can find the paper as a pdf here. If you attended my presentation and are looking for more in-depth knowledge of what we did, take a look into it.
The paper walks you through an application of Agile practices in a traditional environment, where a small group of testers used the practices to succeed with converting their automated test cases to a maintainable new automation approach. The bibliography section will also conclude with the book references I gave.
Thanks to Matt Heusser, Gojko Adzic and Mike Scott on the presentation, Lisa Crispin, Brett Schuchert, Stephan Flake and Gregor Gramlich reviewed the paper, thanks to them, too.
My Definition of Done
Over the course of the Agile Testing Days I noticed that the definition of “Done” is troubling for Agile teams. It was not only raised in the Tutorial I took, but also on several key note speeches and there were several presentations that also dealt with it. Personally I was wondering why. During the last three days I thought “Done” simply means that you can make money with it. Sounds like an easy definition, isn’t it? If your customer or product owner is not willing to pay money for the user story implemented, it’s not “Done”. Period.
But today I came up with another, easier definition of “Done”, which was strikingly easy to realize in the end. For the definition let me put together several influences from Elisabeth Hendrickson, Adam Goucher, Kent Beck, and many more to a remarkably easy combination.
Elisabeth Hendrickson told me that “Done” simply means implemented, tested (or checked as Michael Bolton defined it) and explored. Very easy, isn’t it? But there’s more to it. Implemented means it is designed, and written in probably some programming language, right? Does this make sense? Considering my visit at the Software Craftsmanship Conference in February I noticed that it seems to be accepted – at least for the crafts people I met there – that implementation means to do it using Test-Driven Development. There was no discussion and arguing about this, it was simply taken as a fact and it was done in a TDD-style. But, remembering back Kent Beck TDD means Red – Green – Refactor, right? A three part meme, oh another one. Great.
The next term in Elisabeth Hendricksons definition of “Done” is tested (or checked). Considering current approaches to get the customer’s requirements using an Acceptance Test-Driven Development style, I would like to coin tested to mean, that we agreed on examples to implement and these examples serve as acceptance criteria in an executable manner. So, actually tested here means, that the code was able to pass the acceptance tests which were agreed upfront onto and were elaborated maybe by the testers on the project to also contain corner cases, which were missed. On Monday Elisabeth Hendrickson taught me, that ATDD can be thought of as Discuss – Develop – Deliver. Gojko Adzic corrected this over twitter to Describe – Demonstrate – Develop. But what do we have here? I claim that tested (or checked) here refers to ATDD style of development, which is itself again defineable as a tricolon itself. So, wrapping up, we have
- Done is Implemented using TDD, Tested via ATDD and Explored
- Implemented using TDD is Red – Green – Refactor
- Tested via ATDD is Discuss – Develop – Deliver
(Forgive me Gojko, but I find Elisabeth’s definition more intuitive to remember.)
Oh, I got one more. Explored clearly refers to Exploratory Testing. It actually might be a coincidence, but Adam Goucher came up with a definition of Exploratory Testing today in a tricolon manner, too: Discover, Decision, Action. Sounds reasonable to me. During Exploratory Testing we discover information about the product which we previously did not know. Based on the information we decide what to do about this. Michael Bolton uses to ask me here: “Problem or not a problem?” so that I can decide, what to do next. Inform the next test executed about what to do. After that we take the next reasonable action based on the information we just recently found out about our product. To make the terminology more cohesive here, I propose to coin explored to mean Discovery – Decide – Act.
So, to wrap it up, just like we have Practices, Principles and Values in Agile, we learn using a Shu – Ha – Ri fashion (thank you Declan Whelan for saving the Agile Testing Days for me by mentioning it), we can define “Done” in this same manner:
- Done is Implemented using TDD, Tested via ATDD and Explored using Exploratory Testing
- Implemented using TDD is Red – Green – Refactor
- Tested via ATDD is Discuss – Develop – Deliver
- Explored using Exploratory Testing is Discover – Decide – Act
Agile Testing Days Berlin I – Acceptance Test-Driven Development
Agile Testing Days Website
Over the past three days I was able to attend the Agile Testing Days in Berlin. It was a great conference and I met up with very interesting people, attending really great talks and keynotes. Personally I hope to get back there next year. I wouldn’t have thought such a great event could take place in Germany with such brilliant people from all around the world attending. Jose Diaz and Alex Collino did a great job putting up the conference. Here is a series of reflecting write-ups about certain aspects of the conference. I decided to bring them up one by one just as I experienced the conference and have a short wrap-up in the end. The first entry in this series is about the tutorial session I participated in on Monday.
August of Testing
This blog posting will serve as a catch-all for the web-pages I left open on my tabs during the last few weeks. In case you’re interested in what I did since the beginning of August, it might be worth a read for you.
Back in July Mike Bria posted an article on Test objects, not methods. His lesson boils pretty much down to one single quote that I need to remind myself of:
Keep yourself focused on writing tests for the desired behavior of your objects, rather than for the methods they contain.
Over the past weekend Michael Bolton came up with a final definition on checking vs. testing. Lately I argued whether or not we need a new name for testing. Michael Bolton was able to explain to me in this post that we do not need a new name, but a proper definition. This definition in combination with Perfect Software gives you pretty well insights into what to expect from testing and what to expect from checking. Adam Goucher summarized a talk from Chris McMahon at the Agile 2009 conference with the following insight:
The only reason to automate is to give expert testers time to manually test.
Ivan Sanchez stated that there is another area where a term got lately abused with vague definitions. The term is “done” and he calls for stopping redefining and redefining it. Basically my experience shows that the same is pretty much true to a lot of words in our business. Lean, Test, Done, Agile, …. maybe you have more of these for the comments.
On my way home today I thought over a job profile I read over the past month. Basically when transporting it to my company I would include “You know the difference between testing and checking” in the skill list.
Finally Matt Heusser put up another test challenge. What I loved about it was the fact that I could more or less directly relate the lesson to my daily work. So it’s nothing really artificial if you build the bridge between the abstract challenge and your real-world project. Oh, and of course, for a geek like me it was fun. Wonder, if I could do the challenge in the real-world, once. I love lightsabres. And as a black-belt member of the Miagi-Do school I had to take that challenge, of course. Feel free to exchange some thoughts on the challenge with me if you have taken it.
Interview with Gerard Meszaros
A while ago Matt Heusser asked me to help him out with some interviews. Today InfomIT announced the interview with Gerard Meszaros that I helped with. It’s title is The Future of Agile Software Testing with xUnit And Beyond and I’m glad that I could be of some help there. If you haven’t read the book from Meszaros, make sure to order it. It’s not just covering unit testing if you’re reading between the lines.
XML Unit: assertEquals and fail
Some weeks ago I was challenged at work that led me to invent a unit test framework for xml files. After some basic first steps I decided to introduce the approach on my blog as well.
Motivation
Why did I need a unit test framework for xml files? Easy enough. Our product has an intense configuration consisting of several xml files for the customization part. On the current project I’m working on there is a configuration file which consists of about 18 Megabytes of customization in one xml file. Most of our other tests use to start up a full system (taking a bit more than 60 Megs of main-memory database in combination with oracle database persistence) and exercise the whole chain all through the system. Initially I tried to write a unit test using JUnit and JDom, but failed to have it executed in my IDE while the 18 Megs were trying to be loaded with an OutOfHeapSpace exception. Brian Marick had pointed out Assert { xpath } from Ruby some weeks ago and I started to consider this as an alternative. After realizing that nearly no one knows Ruby at my company and there would be drawbacks considering the way our company uses the Revision Control system, I forgot about these alternative.
Then I reminded myself on an approach using xsltproc and some stylesheets. Usability for our Continuous Integration framework was an issue, therefore I decided to produce an output similar to the JUnit XML reports, so these can be directly hard-wired. This blog entry will describe the first few functions that I built: assertEquals and fail. If there is a demand for follow-ups, I will try to come up with an approach similar to the Test and the Suite classes of JUnit. In the end a test runner will also be needed, which is currently handled by Makefile dependencies.
Continue reading XML Unit: assertEquals and failCustomer relations to beautiful code
What is Quality?
Quality is value to some person.
I would like to show how to apply this often-cited quote from Jerry Weinberg in order to distinguish whether or not to code beautifully (or elegantly) or build a mess. In order to start I will distinguish between externally perceived quality of your software and internal structural quality of the code.
The externally quality of your software is the factor how the end user perceives your software. Bulky user interfaces with complicated workflows can be valueable to someone, most prefer easy to use user interfaces with easy to learn or even intuitive workflows. Therefore your end user will not care much about how beautifully you coded your application using test-driven development or not, using latest design patterns or not, refactoring as you went to ship it. Isn’t it?
No, it is not. Here again it depends on the context of your software and it’s use and the situations your users want to use it. If the business model has a high coupling to laws like tax systems and these laws changes, it actually may be the case that a product of high quality today becomes a mess tomorrow, since I have to recalculate the taxes myself using a calculator. Since quality is timely value to some person quality may disappear if it is not maintained.
Here comes internal quality into play. If you have built a mess with your code, you make yourself unable to react to market or law situations. Ward Cunningham, Martin Fowler and others called this concept Technical Debt. Sure, it might be that you can overcome this situation with another mess, thereby disabling you for the next adaptation you are going to make. In the end you are running into the situation where your code base can no longer with todays needs, since you brought in flaws in yesterday urgency.
Alistair Cockburn came up with a two-phased approach to describe this situation: The Cooperative Game. The initial goal of the game is to deliver working software. This related to the external quality of your software as it is perceived by the customers today. The second goal of a cooperative game in software development is to adapt the system to tomorrows needs. This related highly to the internal structural quality of your code base. No end consumer will take care of this – today – but he will be unhappy to pay for your Technical Debt in terms of later delivery, higher costs tomorrow.