As pointed out earlier, I wanted to work through a test for ParkCalc using templates. As this did not seem to work, and I got no clue why, I gave up and postponed it. Today I was able to take a step back with a fresh mind and approach the problem with a different focus. So, here is the template solution for ParkCalc automation.

Templates

Now, after having run the tests – you did run them, now, didn’t you? – we can see that the main difference between the two tests currently lies in the reporting format of test cases that fail and pass. For the data-driven tests I get

7 tests total, 4 passed, 3 failed

while I get for the keyword-driven tests

29 tests total, 19 passed, 10 failed

Personally, I like the latter format more, as it tells me how many problems in the application based upon my few tests exists. In the former example I’m left to count what I have, but that could be just my personal bias. We’ll take a look back after having refactored the tests to tests using templates.

So, first of all we will copy the data-driven tests to a folder called templated tests and add them to git. Then we open the functional tests. The first thing we notice is that there is serious repetition in the test cases itself. In all five tests there is the keyword mentioned explicitly. A template can help us get rid of this. Let’s consider the Valet Parking tests first, and rewrite them as pointed out by the Test-templates documentation from Robot Framework

Valet Parking Test  [Template]  Valet Parking
  @{FOR_ONE_HOUR}    $ 12.00
  @{FOR_FIVE_HOURS}  $ 12.00
  @{FOR_ONE_DAY}     $ 18.00
  @{FOR_THREE_DAYS}  $ 54.00

A false-negative

Now, if I run these tests, I get some unexpected output:

Valet Parking Test                                                    | FAIL |
Several failures occurred:

1) Page should have contained text '$ 12.00' but did not

2) Page should have contained text '$ 12.00' but did not

3) Page should have contained text '$ 18.00' but did not

4) Page should have contained text '$ 54.00' but did not
------------------------------------------------------------------------------

All four tests for Valet Parking are failing. By manually checking these, I notice that this is indeed a failing test which should pass. So, obviously, I made a mistake. Since we copied the tests simply from the data-driven tests, I suppose the problem is also in that test suite there. Let’s run them and inspect the log file.

From the log.html I can see, that just one of the four tests for Valet parking is actually exercised. Putting my brain cells together over the remaining tests, I can see, that the remaining tests stopped executing when the first error was reported back. Inspecting the resource.txt file from the data-driven tests, I can see, that the submit button is clicked twice by the keywords: once in Park Calc and once in Calculated Cost Should Be. Since the Parking Lot is reset when the page is loaded upon submission, we get unreliable results. This can happen, it’s ok, but let’s learn from this, that we should be suspicious about the outcome of automated tests. So, we fix the problem by removing the submit button click in the Calculated Cost Should Be function. Actually, we could also remove it from Park Calc, but my personal preference is here to leave Park Calc for the calculation, and leave the other function for the cost calculation.

...
Calculated Cost Should Be  [Arguments]  ${expectedCost}
    ${actualCost} =  Get Text  ${COST_ITEM}
...

When executing the tests, we see that we partially fixed the problems. The tests still stop executing when the first problem is encountered. Unfortunately I couldn’t find a better way but to move the built-in keyword Run Keyword And Continue On Failure to the tests directly, thereby obscuring the data-driven tests heavily.

Valet Parking Test
    Run Keyword And Continue On Failure  Valet Parking  @{FOR_ONE_HOUR}    $ 12.00
    Run Keyword And Continue On Failure  Valet Parking  @{FOR_FIVE_HOURS}  $ 12.00
    Run Keyword And Continue On Failure  Valet Parking  @{FOR_ONE_DAY}     $ 18.00
    Run Keyword And Continue On Failure  Valet Parking  @{FOR_THREE_DAYS}  $ 54.00
...

The changes can be inspected here.

A better solution

So, having the data-driven tests obscured, I want to push the templated tests even further by now. We first of all fix the bug regarding the doubly clicked submit button similarly to the way we dealt with it in the data-driven tests.

Since templated tests are automatically run in continue on failure mode, we can avoid the usage of the built-in keyword in these tests completely. First of all we remove the keyword in the Park Calc keyword where the Calculated Cost Should Be keyword is called:

Park Calc  [Arguments]  ${parkingLot}  ${entryDate}  ${entryTime}  ${entryAmPm}  ${exitDate}  ${exitTime}  ${exitAmPm}  ${expectedPrice}
...
    Calculated Cost Should Be  ${expectedPrice}

And then we run the tests, and see that just the Valet Parking tests which we converted to templated tests are now reporting multiple errors. Let’s convert the remaining suite, and compare the results with the data-driven tests.

Having run the tests and comparing the results, we see that the outcome of the data-driven tests and the templated tests is the same. Since the templated tests look prettier and more easily to maintain, I get rid of the data-driven tests for now. They don’t add real value to my purpose any more.

The resulting source tree can be checked out here.

Finally, I hope to conclude the Robot Framework part of this series here.