At CAST 2011 Paul Holland held a presentation on how to organize a peer conference on software testing. Paul gave myself some great pointers for the upcoming organization of the GATE workshop.
Paul defined what a peer conference is. His definition is based on the LAWST format of conference – the Los Altos Workshop on Software Testing which he considers to be the grandfather of peer conferences. A peer conference is a small, invitation only gathering, maximum 15 to 25 people (depending on the facilitator’s ability). The usual seating is in a U-shape.
On a peer conference every participant is also a presenter. Who can talk and when is very structured so that everyone can contribute based on their own preferences. Typically the presentations are based on first hand experience. Every participant is expected to participate in the discussions going on. Observers are permitted, but they do not sit at the main table. Presentations are more like telling a story than attempting to instruct or present theoretical material. A peer conference can be as small as within a city or a company, or it could be up to an international conference like the Writing about testing conference I attended earlier this year. On the bottom line, a peer conference is a fantastic way to learn from your peers.
On the history of peer workshops, Paul referred to the history of the Los Altos Workshop on Software Testing. Cem Kaner, Elisabeth Hendrickson and Brian Lawrence created LAWST in 1997 getting together with all the problems they had with capture & replay tools back at that time. They found after the workshop that no one was really applying successfully capture & replay tools. Paul referred to the list of likewise workshops like WHET, WTST, STMR, TWST, WOPR, STIFS, WREST, WWST, AWTA, POST, and many others (like GATE). The rules created by the LAWST meetings and the subsequent spin-offs since them include to distinguish between clarifying questions and the open season. There are colored cards which indicate whether you want to continue on the same thread of question (yellow), have a new thread of discussion (green), or a burning issue with what was just said (red). Facilitation is happening just-in-time.
Paul described how to get started. The people involved include the primary organizer or the organizing committee. They work with everyone below, keep track of the overall timing like the call for proposals, acceptances, etc., and the primary organizer is also the content owner. The content owner sets the theme, the call for proposals, and selection of participants. The content owner also sets the order of presentations during the conference. Paul made clear that you can use all of his suggestions, but may also perfectly avoid it. Depending on the context for your workshop you might end up doing different things, but this worked for him in the more than 30 workshops he helped facilitate and organize.
For the facilities Paul presented a guideline regarding the rooms. A suitable meeting room is usually enough. The facilities prime should take care of that. He also manages the food like breakfast, lunch and snacks. Regarding security, audio and visual, the room layout and the facilitator’s chair should be taken care of. Also for people staying in town organizing evening activities to conclude the workshop day is recommended.
The facilitator controls who is allowed to speak during the entire meeting, and when. He needs to have the full support of the content owner. The facilitator must have the authority to remove non-compliant individuals and should not personally participate in the discussions. In the end without a good facilitator, the meeting can quickly get out of control.
Paul went on with the economics of a peer conference. The official LAWST way of doing it is to ask attendees for donations. You may also get a company to host the conference – typically for a guaranteed number of seats at that conference. Pay for it yourself (often combined with the first option). Sometimes you can ask one of the attendees’s companies facilities. AST offers financial support for peer conferences as well based on a limited budget, but it doesn’t hurt to ask. Finally, you can charge a cost sharing fee, which may exclude the organizers, but you will also give up AST-based funding at that point.
Paul explained that the IP agreement is crucial to any peer conference – any conference in fact. Every peer conference should use an IP agreement to clarify the rules of ownership of whatever is presented and developed at the conference. Cem Kaner developed an IP agreement which is quite thorough. Anything presented at a peer conference should be allowed to be used by anyone attending.
Paul suggested a schedule based on his experiences with WOPR. 26 weeks before the conference they find a content owner and a host. The theme, dates and location are announced between 15-25 weeks. 14 weeks before the conference the call for proposals is set up. Early submissions are monitored twelve weeks prior to the conference. Four weeks before the conference new attendees get assigned a mentored so that the contributions are actually experience reports. In the final two weeks a final mail on logistics is sent out with the location, timing of pre-confernece dinner, start time of the conference, food being offered, etc. Finally have a great conference.
Maybe the most influential part of a peer conference is the facilitation, Paul explained. The rules work very well. That is why the rules should be followed strictly – even with strong opinions in the room. On just-in-time facilitation Paul explained that you don’t need to explain all the rules at the beginning. Instead explain them as they come up, and use thumb votes and other simple solutions.
Ask for the help of participants to recognize rat holes. Let the group know that they control the direction of the meeting with their energy levels and their questions and comments. Maintain control of the group, so don’t allow anyone to talk out of their turn. The facilitator handles the stack of comments and questions floating in. People who talk less should get called upon before frequent talkers. The stack should not be managed in a FIFO style. Before moving to a new thread, exhaust the thread (and any sub-threads occurring). Try to finish with a strong participant – which might not be easy to do. The speaker is always allowed to respond to any comment or question even if not directed at them – it is their presentation, after all.
2 thoughts on “CAST 2011: Build a deeper community of practice – How to organize a peer conference”
Markus – just a note that I have been running what I’ve called “peer conferences” for about twenty years. I mentioned the term to Bruce Eckel at the 2003 AYE Conference and James Bach the following year and both of them seemed to have adopted it for their small conferences. The Conferences That Work design creates a “structured unconference” for any peer group that has a common interest, so it has a wider scope than just the field of software testing.
My peer conferences are also small, but they’re not invitation only, and I don’t require everyone to offer a presentation, though the conference is specifically designed to support and encourage participation. I also use a set of six explicit ground rules that everyone agrees to at the start of the event.
My book on peer conferences Conferences That Work: Creating Events That People Love was published in 2009, and is surprisingly (to me) popular.
Anyway, I’m not trying to stake a claim to the provenance of the term “peer conference” or opine how it should be defined; just contributing a little to the history of the term.
I remember coining the term peer conference around 2003 as a replacement for “LAWST-style conference” that we had been using since 1997. You say you spoke to me about this at AYE? I don’t remember that, but then again, it’s the sort of thing I might well not remember.
The use of the term peer conference in the CDT community definitely comes from me, though. I was actively campaigning for this to contrast with “exhibition conferences.”