A while ago, I called for some participation on the state of our craft. I promised back then to present some intermediate answers in late January. Here they are.
Up until know 183 people have filled out the survey.
The first question in the programmer’s survey is on the particular programming the programmers used.
On the usage of TDD, I received some interesting answers so far.
43% use TDD regularly, 20% at least once per week, 17% about once per month. 17% never use TDD, and 3% don’t even know what it is. This surprised me, as I thought that fewer people use TDD on a regular basis.
On refactoring the picture seems a bit better.
64% use refactoring several times per day; 24% use it at least once per week, 8% about once per month. 3% never use refactoring, while 1% does not know refactoring at all. More than 90% of the people who took the survey at least know about refactoring, and use it on a regular basis – ome more often than others. This didn’t really surprise me, as today’s IDEs – besides ones like XCode or Visual Studio – ship with support for automated refactorings. Probably that’s why so many use them regularly.
The final question in the programmer’s survey referred to the SOLID principles.
48% consider it part of their work, while 32% think about some of the letters from time to time. 25 don’t need SOLID, and 19% wondered about why I wrote it all uppercase. Surprisingly this number is also higher than the numbers I had thought of before sending out the survey. So I was again surprised.
The tester survey was mainly motivated by Lee Copeland’s Nine Forgettings. So far 104 people have filled it out.
First of all I wanted to know about the experience in the field. I was interested in the experience of the people filling out the survey, and how this might correlate with the different viewpoints.
42% have more than 10 years of experience, 37% more than 5 years, 7% 4-5 years, 7% 2-3 years, 6% of about one year. 2% stated they started with testing just recently. This reflects a huge level of experience.
Next thing I asked for some testers in the past. Reflecting on Lee Copeland’s presentation, I was interested in how many people actually know these testers.
90% know Michael Bolton, 88% James Bach, 78% Jerry Weinberg. Michael Fagan, who introduced inspections, know 33%, Dave Gelperin, who authored the IEEE-829 standard, know 13%, and Bill Perry, who founded SQE, know just 8% about. I think this reflects some worries from Lee, but I was still surprised by the knowledge about some of the past heads in software testing.
On the last book read by the tester, I collected together a bunch of books, which I found helpful in the past.
19% have read some other book, 18% Lessons Learned in Software Testing, 18% Agile Testing, and 13% Perfect Software… and other illusions about testing. On the other answers, one thing struck me. There were some people pointing to books as Growing Object-Oriented Software Guided by Tests, and Working Effectively with Legacy Code, or Continuous Delivery. I don’t considered these as software testing books as to my understanding. I think this reflects the ambiguity that comes with the term software testing, but that could be just my interpretation.
Another thing struck me. 4% claimed they never have read a book on software testing.
On the book recommendations there were a few interesting points. At least two participants claimed that they read any of the ISTQB books last, and would not recommend any book on software testing. A lot of recommendations go out for Lessons Learned in Software Testing. Interesting in my data is also, that most of the participants who don’t recommend a book, have read books like any of the ISTQB books, The Art of Software Testing, or Managing the Testing Process.
When I try not to overrate the data, I would claim that we do not have the same understanding of what good testing consists of – given the variety of responses I got.
On the programmer’s survey, I think my data is biased. I might have overestimated my reach with the survey here. I am quite sure that these data don’t reflect the state of our software development craft.
On the tester’s survey, I think my data is also biased. People reading my blog might actually be influenced by similar sources as I am. This means the context-driven and the Agile testing world. That’s ok, but I hoped for a variety of responses here as well. I seem to have got such responses by accident, but would like to see more people filling out this form to gain more insights, maybe.
That’s it for now. I hope to get some more responses, but don’t put too much faith into it.
5 thoughts on “Responses from the programmer and tester surveys”
How do these survey results generalize to anyone beyond the specific people who answered? I think the answer is: you have no idea. How did you control for different interpretations of each question. I think the answer is: you didn’t.
I respect you, Markus, but surveys like this are an attractive nuisance: http://en.wikipedia.org/wiki/Attractive_nuisance_doctrine
I take it as a learning opportunity for myself. With the surveys I tried to find out data about claims that my colleague and Lee Copeland made. These were based on “hunches”.
Validating the set of responses is fairly one thing that I oversought. Lesson learned.
I found an error in the survey for testers. You ask for names you recognise. If you do not know anyone, the form refuses to send to the backend.
The programmers survey is about technology, the testers one is about persons or books you know or read by you. Strange.
The main difference is that I had different sources for motivation. My colleague for programmers, Lee Copeland for testers. I address different claims with each.
From testers experience, we can who are the ones roaming in blogs and other sites – seems like it takes a veteran to understand that reading other people blogs and articles is useful.
Newcomers just don’t understand that they lack knowledge…
:-( :-( :-( So sad…