After having proposed the basic idea for Matthew Heusser’s latest testing challenge, I would like to present the refinement based on Matt’s answers. Additionally I would like to make sure to be able to come up with a third part on it after getting back some feedback just before the deadline. In addition please mind that I have been quite ignorant of the answers and reactions on it so far, despite one answer from Philk, which I found funny to read – and mind-opening when comparing to the company I’m working at.
The Five minute challenge
The high level test approach I would like to choose here consists of some main charters:
- Explore signal sending
- Explore conversations
- Explore networks
- Explore the remains: Drop-down filters, history, RSS feed
Some more explanations will be useful here. So let me try to stress these out in a bit more detail.
Explore signal sending
Some first few ideas I would draft on my notes for the exploratory tests on this item include sending a regular message, sending a message with 140 characters, trying too few, too many characters. Try replying all sorts of messages, noticing if the counter with the remaining characters is reduced properly. What happens if I try to reply to a message that already has 140 characters? Is there a configurable bell sound for notifying me? A visual aid? Is there a visual aid, when the number of remaining characters drops below a certain threshold? Highlighted messages, self-highlighting, private messages, are they really private? Will the background be great or gray? May I delete private messages, too? (Like indicated by the trash symbol in the view) What about timestamping behavior? Are they properly updated? Are they updated online or just on refresh?
After stressing these few tests out, I will be in the position to think of what to automate regarding this feature. The automated tests would include sending a regular message, failing to send a too short and too long message, message replies and private messages (create, read, delete) at least. If the exploration uncovers other hidden risks these tests would be included there, too. I would try to integrate as good as possible in the already existing testing approach, which is probably based on Watir or Selenium, as much as possible. If impossible at all, I would need to come up with an self-crafted driver for the REST API, create users, etc. I would try to get in touch with the developers as much as possible on their help on this.
Edit a page, add a comment, edit your own comments. Edit my profile, tag my profile, tag the profile of someone else. Delete a tag of my own profile and delete a tag on another profile, a tag someone else set. Check that I can’t edit another user’s edit, check that I can edit another user’s page and that raises an conversation event. Watch a page and have another user edit it, comment it.
Tests to automate include editing a page, adding a comment, editing a comment, profile edit, profile tagging, profile tagging of someone else, deleting tags on own profile, another profile, another user’s tag on any profile. Edit another users page for a conversation event, page watching and having it edited and commented. While I’m at it, checking that I can’t edit another user’s page should be possible, too. So I would basically automated all of the above.
Follow people, unfollow people, follow people and send a message, follow people and check their messages. Conversations were already explored in a different context. Maybe switching networks in between could yield some more information, but is up to the exploration.
Automation should do follow people, unfollow people in combination with sending some messages in between – at least.
Explore the remains: Drop-down filters, history, RSS feed
Open all drop-down filters, check spelling, consistency regarding the interface representations, properly overlay with the underlying web side. Scroll through message histories, older, newer, going back to newest. Subscribe to rss feed and watch some notifications appearing. The wrench symbol indicates some advanced setting on the window in the right corner. What happens if I click there? What can I explore around there? What happens if I resize the window? What does minimize plus restore do to the view? When hitting the close button, while having some typed text in it, is there a question asking me if I want to throw away my changes in the edit box?
That’s pretty much what I can forsee now at the moment so far. The general automation approach is interleaved with the actual activities I would like to do during the iteration. Two weeks are a pretty big deal of time considering this feature, and I doubt it will be the only one in that iteration. Therefore there will be some need to reduce the actual testing of the widget down, so that time permits further testing of other stories. Basically I would try to timebox the activities for the testing – not the bugfixing – around this to two days, if possible. With good knowledge on how to automate tests for this beast, it should be possible, but that will depend on the team. Maybe I could by then also help our developers on the unit tests for this, but as time unfolds there will be more informations on these questions.