ParkCalc automation – Getting started

This week Gojko Adzic wrote about the anatomy of a good acceptance test. After having read his elaboration, I remembered how I came up with the preparation for the EWT19 session some weeks ago. We used RobotFramework to automate tests for the Parking Lot Calculator that we asked Weekend Testing participants a few weeks earlier with manual Exploratory Testing. To get testers started we provided them with three examples that I prepared before the session. We then asked testers to automate their tests for the ParkCalc website based on one of the examples we provided. Here is my write-up how I came up with the examples, and what I had in mind.

Introduction

First of all, we had picked the ParkCalc from the Gerald R. Ford International Airport as product to be tested. Adam Goucher was kind enough to provide a copy where we could run our test against without bringing down some production system of an airport. That said, the parking rates on the official airport page served as an oracle for the correct prices.

Since RobotFramework provides great abilities to test this application when used together with the Selenium Library for Robot Framework, we asked the participants to install the tools before joining our session. Additionally we were happy to have Pekka Klärck, the main developer of the Robot Framework project, with us.

I won’t get into details on how to install Robot Framework or the Selenium Library. There is a great introduction for Robot Framework on their wiki, as well as a demo for the selenium library which should get you started. In case you run into problems, please drop the authors a note. They will be happy to help you out on this one.

The examples we provided to the participants can be downloaded here. There are three examples included in the .zip file, called calc1, calc2, and calc3. The examples are based upon the selenium library demo mentioned above.

The first example

So, I needed to get a first test started. We explored the product a few weeks earlier, and focused mainly on problems related to date time calculation. The mission now was to get automation in place, since all of these bugs were going to be removed from the code. Therefore a good regression test suite should serve the purpose to ensure the underlying business rules will still be met with the new code. A perfect part for test automation.

After fiddling a bit with the id’s in the form and getting a first understanding how robot framework works, I ended up with a straight-forward test that had everything inlined. Here is the essence of the test:


Basic Test
    Open Browser  http://adam.goucher.ca/parkcalc/  firefox
    Set Selenium Speed  0
    Title Should Be  Parking Calculator
    Select From List  Lot  Economy Parking
    Input Text  EntryTime  01:23
    Select Radio Button  EntryTimeAMPM  AM
    Input Text  EntryDate  02/28/2000
    Input Text  ExitTime  12:34
    Select Radio Button  ExitTimeAMPM  AM
    Input Text  ExitDate  03/01/2000
    Click Button  Submit
    Page Should Contain   (1 Days, 23 Hours, 11 Minutes)
    [Teardown]  Close Browser

Looks like a good test, doesn’t it? It expresses that parking in the economy lot from 02/28/2000 01:23am to 03/01/2000 12:34am calculates to a duration of 1 day, 23 hours, 11 minutes. Wonderful.

Of course, there are several problems with this test. It uses no variables, it does not abstract from the implementation of the test script to the business rules, and it has many values hard-code which make the test messy to read. At work, I would consider this one to be a spike solution and throw everything away, not even submitting it to the source code repository. To be honest, the first test serves the purpose to learn about the tool and get enough knowledge about the tool domain to get the real tests started.

A data-driven test

Now consider the parkCalc3 test. Here is the test:


Basic Test
    Park Calc  Valet Parking  05/04/2010  12:00  AM  05/05/2010  12:00  AM  $ 42.00

This expresses that for Valet Parking from 05/04/2010 12:00am until 05/05/2010 12:00am a charge of $42.00 will be calculated. Regardless whether or not this is correct, the application prints this one out.

Now, does this test look any better compared to the former one? First of all there is a single keyword now driving the application from the tests. Taking a look into the resource.txt file in the same directory as the test itself, shows up the following implementation:


Park Calc  [Arguments]  ${arg1}  ${arg2}  ${arg3}  ${arg4}  ${arg5}  ${arg6}  ${arg7}  ${arg8}
    Open Browser  http://adam.goucher.ca/parkcalc/  firefox
    Set Selenium Speed  0
    Title Should Be  Parking Calculator
    Select From List  Lot  ${arg1}
    Input Text  EntryTime  ${arg3}
    Select Radio Button  EntryTimeAMPM  ${arg4}
    Input Text  EntryDate  ${arg2}
    Input Text  ExitTime  ${arg6}
    Select Radio Button  ExitTimeAMPM  ${arg7}
    Input Text  ExitDate  ${arg5}
    Click Button  Submit
    Page Should Contain  ${arg8}
    Close Browser

It seems that we simply have applied some extract method to the first test, and ended up with the same script here. The browser as well as the url are hard-coded which may make future adaptations tricky – in this case it’s just the extraction of a variable. The names of the parameters are awkward – and indeed I confused them when I wrote this example down. When a new parameter is introduced, we’ll be pretty much doomed with this test. At least we should then take action and rename those parameters to that we won’t be confused any longer.

Finally a good one?

So, let’s take a look on the second example.


Basic Test
    Open ParkCalc
    Input Parking Lot  Short-Term Parking
    Input Entry Date  12/31/2010  12:00  AM
    Input Leaving Date  01/01/2011  12:00  AM 
    Calculated Cost Should Be  $ 28.00
    [Teardown]  Close Browser

This test checks that for short-term parking from 12/31/2010 12:00am until 01/01/2011 12:00am a cost of $28.00 will be calculated.

Now, is this one any better? In terms of good tests, it isn’t. There are some keywords in use in this test, but the names still reflect the nature of the user interface. If these tests should serve as executable specifications we surely may want to get rid of the inputs and of the closing of the browser. It shouldn’t bother us at this time. Taking a look on the resource file include in the second examples shows us the following implementations of the keywords:


Open ParkCalc
    Open Browser  ${PARKCALC URL}  ${BROWSER}
    Set Selenium Speed  ${DELAY}
    Title Should Be  Parking Calculator

Input Parking Lot  [Arguments]  ${lot}
    Select From List  Lot  ${lot}

Input Entry Date  [Arguments]  ${date}  ${time}  ${ampm}
    Input Text  EntryDate  ${date}
    Input Text  EntryTime  ${time}
    Select Radio Button  EntryTimeAMPM  ${ampm}

Input Leaving Date  [Arguments]  ${date}  ${time}  ${ampm}
    Input Text  ExitDate  ${date}
    Input Text  ExitTime  ${time}
    Select Radio Button  ExitTimeAMPM  ${ampm}

Calculated Cost Should Be  [Arguments]  ${cost}
    Click Button  Submit
    ${actual} =  Get Text  xpath=//tr[td/div[@class='SubHead'] = 'COST']/td/span/font/b
    Log  Actual costs: ${actual}
    Page Should Contain  ${cost}

So, there seems to be some structure to it. This looks a bit better, but still not optimal to serve as executable specifications.

Which one is better?

Now, as all the tests indicate a simple basic example, and we want to dig deeper on the business rules, we now should decide which one to use in order to start off our next few tests. As mentioned before, the first example is a spike solution from my perspective. It simply serves the purpose to learn something about selenium keywords, and how to get started with the new tool domain. It’s surely the best option to forget completely about that.

The third example is a data-driven test. It’s current implementation of the keyword used is a bit sloppy, but it can be used to express the business rules in a simple table format. So, it might be worth taking a closer look into it.

The second example is a keyword-driven test. It’s current implementation is sloppy, too, but it can be improved. Additionally by combining new keywords this test may also be used for bad path tests in the future maybe. So, it might be worth taking a closer look in it, too.

From my perspective it’s a matter of personal taste whether to use the data-driven or the keyword-driven example in this case. I preferred the keyword-driven test, but I can also see the benefits of a data-driven test in this example. Therefore I will spend in a follow-up post some time to automate some tests in a data-driven format, and in another follow-up post I will show how I work through a keyword-driven approach. Meanwhile you may want to try yourself on improving the tests as shown – maybe with some of the insights you get from Gojko’s blog entry mentioned earlier. Stay tuned.

  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • LinkedIn
  • Google Bookmarks

13 thoughts on “ParkCalc automation – Getting started”

    1. Thanks for pointing out to the articles from Dale. These are indeed great introductions into the topic. I am also thinking about working through the same product with another framework, but I’m not sure, if this one will be taking too much time for me to get through. :) Time is always a constraint.

  1. Hi Markus,

    First off all I like very much the idea that you choose an application to test and then try some different approaches.
    I am open to this kind of scripting where not much time is spent, no complex frameworks made in rush and need debugging, and there is I think some good results in using it.
    Cool application to test. I tried it a little bit and used perlclip to generate me a string of 1 million chars and inserted in each of the boxes and press “Calculate” and got this:

    Request-URI Too Large

    The requested URL’s length exceeds the capacity limit for this server.

    I tried also selecting the same date and time and press calculate and saw cost “2.00” but I think this was already observed.
    But anyway I think is very good idea to use real applications.

    Sebi

    1. Thanks for your hints Eusebiu. In fact, some time earlier we derived a testing flash-mob over twitter based on the original page. On that weekend we used a mission for manual testing of the application, and found several problematic areas there.

      The idea behind the EWT19 test automation mission was, that the management from the airport got the attention based on our previous manual testing efforts that there’s something wrong with the page, so they want to exchange it. While doing so they want to bring in automated tests that shall check the correct behavior. Here comes RobotFramework and the selenium-library into play, as they greatly support web applications like these.

      So, we’re currently trying to achieve a readable specification for the parking rates calculator. We have basically all information, and this is what this little series is all about. There are other sources that deal with the ParkCalc flash-mob. If memory serves, infoq had a summary on it, too.

  2. Markus,

    Fantastic post. Inspirational, really. By using such a clear example and walking through your thought process, you’ve provided a superb explanation. Keep posting blog posts like this! You’ve hit on a formula here that really works.

    While I’m not saying you’re the first testing blogger to use this kind of example-driven formula, (I’m thinkin’ bout you Mr. Heusser), I’d be thrilled if other bloggers reused the template you’ve created here for their posts. I, for one, have made a mental note to do just that. Perhaps by posting a blog that follows up on youra here and describes a couple approaches of how pairwise and/or combinatorial test design approaches could be used to help testers identify a highly efficient set of test inputs for these test cases.

    Justin

    Justin

  3. Markus,

    Suggestion: include a screen shot of the Park Calc app.

    Again, great post. I’d recommend it to other bloggers as a classic example if “how to write an effective blog post to clearly communicate pragmatic, actionable testing techniques.

    Justin

Leave a Reply

Your email address will not be published. Required fields are marked *