Functional Testing: Testing MultiDoc
Our First Functional Test
Functional Testing with MultiDoc
We're going to look at some example techniques for functional testing. The specific examples use MultiDoc, a multi-tabbed text editor that is the example application from chapters 3-6 of IronPython in Action.
Although the specific code I will show is specific to IronPython and Windows Forms, the basic principles apply to most GUI toolkits.
There are basically two different techniques for functional testing.
- In process testing (with hooks into your application)
- Driving the application completely from the outside as a separate process
A lot of testing tools use the second technique, which can make testing a lot harder - particularly on Windows where even through the win32 APIs there is some information you just can't obtain. Other operating systems may make this easier, but it is still going to be harder than directly interrogating your application about its state. You will still want some 'separate process' tests though - for example to test installers and file associations and so on.
GUIs and the Event Loop
- GUI programming is typically event based programming
- The 'event loop' runs in (and takes over) a single thread
- You provide callbacks for user actions
- We need to invoke actions onto the event loop to simulate user actions
- And then make assertions about application state
- So we need to start the application in another thread
The event loop is called the 'message pump' in Windows Forms. When the event loop starts it takes over the thread on which it is running - so to test the application 'in process' we start it in its own thread. This means that we need to be able to interact with the GUI thread.
Start the Application
fileDirectory = Path.GetDirectoryName(__file__)
executableDirectory = Directory.GetParent(fileDirectory).FullName
self.mainForm = MainForm(executableDirectory)
This is a method on FunctionalTest, which is a subclass of unittest.TestCase and the base class for all our functional tests.
Application.Run starts the event loop and as this is Windows Forms it has to be started in an 'STA' thread.
startMultiDoc starts MultiDoc in the same way it starts normally - by instantiating the MainForm class with the path to the directory it lives in. Once the mainform is instantiated it is set as an attribute on the test instance so that we have access to it from inside our tests.
Create an STA Thread
self.mainForm = None
self._thread = Thread(ThreadStart(self.startMultiDoc))
while self.mainForm is None:
The setUp method is run before every test. This creates a thread for the application and then starts it; waiting for the application to actually start before handing control over to the test.
Invoking onto the GUI Thread
For the test to interact with the application:
All interaction with UI components must happen on the control thread.
All windows forms 'Controls' have an Invoke method that takes a delegate and invokes it onto the control thread. It waits for the delegate to be processed (it blocks) and then returns the result.
To turn a Python function into a delegate we use CallTarget0.
And when the test is finished we close MultiDoc:
Our First Functional Test
mainForm = self.invokeOnGUIThread(lambda: Form.ActiveForm)
# * A window appears
"Main form is not visible")
# * With the title "MultiDoc Editor"
title = self.invokeOnGUIThread(lambda: mainForm.Text)
self.assertEquals(title, "MultiDoc Editor",
This is a very simple test - it simply tests that when MultiDoc starts a window with the correct title appears. You can run this test by starting basictest.bat from the example code.
The test file has the appropriate if __name__ == '__main__' magic so that we can run the test on its own whilst we are implementing it.
The actual test is FunctionalTest/tests/basictest.py. It sleeps for 25 seconds once MultiDoc has started; so you can see that the application is responsive and not merely frozen.
In the next part we write a more advanced test and actually interact with a running MultiDoc.
For buying techie books, science fiction, computer hardware or the latest gadgets: visit The Voidspace Amazon Store.
Last edited Tue Aug 2 00:51:33 2011.