Functional Testing: User Stories
A Human Readable Specification
Before you can start testing you need to know what you are going to test. Extreme programming advocates the division of a specification into individual features called 'user stories'. These describe a feature from the point of view of the user.
Below is the first part of a user story from Resolver One. It is for the 'show bounds' feature - pressing a button on the toolbar will visually show you the bounds of data in the grid.
In Resolver One user stories the user is called Harold...
- Harold starts Resolver.
- He notices the toolbar button "Show Bounds".
- It's not checked and all the cells are white.
- He clicks it and it becomes checked.
- All cells become grey.
- He enters "cheese" at C5 then clicks on C6
- C5 is no longer gray, it's white.
- All other cells remain grey
Using user stories can have several advantages. They work very well with iterative development - iteratively and incrementally adding new features and evolving the ideal architecture (instead of assuming you know the ideal architecture before you start).
In extreme programming we write the functional test for the feature before you write the unit tests or any code. When you run the functional test it will fail, so you know what to work on next... As you implement more and more of the functional test it will get further through the user story.
Once the functional test passes you can move onto the next one. At each stage you have a 'working' application with some functionality. It makes it substantially easier to respond to changes in specification (or the discovery that the features you have already implemented aren't as useful as expected).
The Implementation: Part I
Here is the first part of our functional test that goes with the user story:
# * He notices the toolbar button "Show Bounds".
item = self.getToolBarItem("Show Bounds")
# * It's not checked and all the cells are white.
# * He clicks it and it becomes checked.
As you develop your testing framework with high-level convenience methods it becomes a DSL (Domain Specific Language) for testing your application.
The test framework we use is the Python standard library module unittest. The methods used above aren't a standard part of this module - we've developed them alongside Resolver One to make it easier to test.
The Implementation: Part II
# * He enters "cheese" at C5 then clicks on C6
self.setCellValue(3, 5, "cheese")
# * C5 is no longer grey, it's white.
color = self.getCellBackColor(3, 5)
# * All other cells remain grey
In an ideal world we'd have one line of code for every line in the user story.
There is a lot of detail involved in making the tests less fragile - timing, invoking onto the GUI thread, re-trying unreliable actions and so on. This 'painful stuff' can be abstracted out into the framework. It takes a while, but it makes your framework more stable, reduces spurious failures and eventually makes writing new tests a lot easier.
You can let the functional test drive development:
- Run the test (it fails)
- Write unit tests & code to get past the first failure
- Run the functional test again
- Now it fails at a different point
- Go back to step 2 and repeat
- Until the functional test passes
Inevitably along the way you will find that the original test is underspecified and that there are additional interactions that you need to test.
On the next page we look at the basic techniques used in functional testing, and create a functional test for MultiDoc.
For buying techie books, science fiction, computer hardware or the latest gadgets: visit The Voidspace Amazon Store.
Last edited Tue Aug 2 00:51:33 2011.