Python Programming, news on the Voidspace Python Projects and all things techie.

IronPython & Windows Forms VI


This article has moved.

You can find the whole tutorial series at IronPython & Windows Forms.

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-06-23 00:15:12 | |

Categories: , ,

The Build Cat

emoticon:cat Our full build (functional tests and unit tests) now takes around ten minutes. This is a problem. Smile

Our first take was to try and run the tests in a virtual machine. Our functional tests simulate user input with calls like SendKeys.SendWait. This is very unreliable with VMWare, spurious failures all over the shop. On Virtual PC it works, but is painfully slow.

This isn't such a problem. Because we pair on everything, when we start a build we can simply go and work on the 'other' PC [1].

What's worse is when both pairs end up building simultaneously. Once one pair has checked in, the others have to do an update and a fresh build. To solve this we have introduced the build cat. If you want to check-in, you must acquire the Build cat before kicking off your build. We've called him Mutex. Razz

Mutex the build cat

Oh, and welcome to our intern, Max. Hello Max.

[1]Although we pair, we all have our own desk and PC. It's nice to have a place to call home.

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-06-22 21:25:53 | |

Categories: , ,

IronPython & Code Quality Tools

emoticon:pill At Resolver we would like to integrate some tools into our build process that check our code quality and test coverage.

The most common coverage tool seems to be, last worked on by Ned Batchelder.

That, and all the others that I found use sys.settrace [1]. This works with Python stack frames and so isn't implemented for IronPython. The line number information is still in there, because we get it back in tracebacks. Currently there is no good way of getting coverage information with IronPython. makes very simple use of sys.settrace. It uses the same function for global and local trace functions :

c = {}

def t(f, x, y):
   c[(f.f_code.co_filename, f.f_lineno)] = 1
   return t

Because IronPython does name mangling and dynamic code generation, it's not likely that native .NET tools would be much help.

For code quality tools I looked at the three most common [2] :

All three of these use compiler, compiler.ast, and/or parser. The parser module isn't available on IronPython, so none of these run.

We can run the code quality checks over our code using CPython however. The basic information we want is stuff like :

  • Unused imports
  • Unused variables
  • Shadowing builtins

Our tests ought to pick up errors like using undefined names (we're maintaining a steady 3:1 ratio of test code to production code), but it wouldn't hurt to be warned about these as well.

I had to drop PyChecker straight away, it attempts to execute code to check it.

I tested the other two with the following ridiculous snippet of code :

import foobar
x = 3
int = 6
def fred(hello):
    silly = 1
    print y


PyFlakes reported the following errors (not enough !) : 'foobar' imported but unused undefined name 'y' undefined name 'james'

It hasn't picked up on the builtin shadowing or the unused variables.

PyLint gave back all we wanted, and lots more, so it looks like that is what we'll use. Here's the relevant part of the PyLint report :

************* Module snippet
W:  3: Redefining built-in 'int'
E:  6:fred: Undefined variable 'y'
W:  5:fred: Unused variable 'silly'
W:  4:fred: Unused argument 'hello'
E:  9: Undefined variable 'james'
W:  1: Unused import foobar

8 statements analysed.

When I reported these problems on the IP Mailing List the developers replied that they intended to implement sys.settrace soon, and would see how much work it would be to implement compiler (which requires parser). Cool response !

In fact there could be a pure Python alternative to the compiler module (parts of it anyway). EasyExtend has borrowed the parser from PyPy, and can generate a CST from Python source.

PEP 269 (a pure Python version of pgen, which generates the Python parser from the grammar) has been bumped from Python 2.3 to 2.4, from 2.4 to 2.5, from 2.5 to 2.6... sigh

[1]Unsurprising really, as that is what it is there for.
[2]Well, the only three I knew about; and a brief search didn't turn up anything else. Hmmm... except I didn't try cheesecake, but I think this reuses PyLint.

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-06-22 20:46:52 | |

Categories: , , ,

Pythonwin & Movable Python

emoticon:pencil A user asked on the Movable Python Mailing List asked if it is possible to use the Pythonwin IDE [1] with Movable Python.

Pythonwin itself is usually launched from an executable called Pythonwin.exe. As you might imagine, this fails miserably when we use it with Movable Python, because it is looking for the installed version of Python.

Thankfully most of Pythonwin is implemented in Python code, using the various win32 extension modules. The exe file is just a convenience stub. Mark Hammond came to the rescue and provided a pure Python file that does the same :

import sys
import win32ui
# importing 'intpyapp' automatically registers an app object.
from pywin.framework import intpyapp

# Remove this script name from sys.argv, else Pythonwin will try and open it!
sys.argv = sys.argv[:-1]
# Get the MFC "app" object and boot it up.
app = win32ui.GetApp()

Download a copy of the pywin32 extensions. [2] Copy the pythonwin directory into your movpy\lib directory.

Save the snippet of code above as in the pythonwin directory you just created. If you run this file from Movable Python, the IDE will run. Smile

[1]Part of the pywin32 Extensions by Mark Hammond.
[2]If you download the executable installer you should be able to open this with winzip or winrar to extract the contents.

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-06-22 13:45:50 | |

Categories: ,

Visual Studio Tools for Office

emoticon:computer Today I went to Microsoft Office Developers Conference. It was a long event, but more interesting than I expected and a lot of information to take in.

It was basically Microsoft selling the features of the Office 2007 suite to UK developers by demoing what can be done with it. Most of the 'impressive' features are enterprise level ones, that require setting up large infrastructures for document management across organisations. They demo well, but look very complex to use. Lots of other nice touches though.

The thing that was most interesting, and even potentially relevant to Resolver, was VSTO.

VSTO and its bedfellow VSTA allow you to write Windows Forms applications that are hosted within the office suite applications (and can access the current document and data). Microsoft are attempting to make Office a development platform, and hope that ISV will create applications that depend on office.

This tool, currently in beta, can use any .NET language; yes that means IronPython. It particularly leverages custom task panes and the ribbon, both of which are new in Office 12 (i.e. Office 2007). This makes your application 'native' to office. You can embed controls within documents, but unlike previous techniques for working with Office, this is done on the application level rather than at the document level (read per document).

You ship your application with a VSTO runtime (which probably needs to be licensed) and the user must have the relevant office application installed to use it. Despite the obvious cynical conclusions, it does open up new possibilities for interaction with the office applications. Given that your target market almost certainly has these tools installed and is familiar with their basic UI, it is a new deployment technique; a way into markets currently held in stranglehold by Microsoft.

When version three of VSTO ships it will also work with the major Office 11 (2003) applications but not all of them.

VSTA is intriguing. It is a version of Visual Studio that you license and ship with your VSTO application and allows your users to customise the application. Presumably you control how it can be customised. Not so relevant to us perhaps, but it certainly has possibilities...

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-06-21 23:48:42 | |

Categories: ,

Python Bits & Pieces

emoticon:fish Lots of good (and new) things have been happening in the Python world recently. Here's a quick overview of the few that have caught my eye.

  • cerializer

    This is a safe pickle replacement [1]. This module allows you to safely save and restore (serialize) Python data structures including classes. With the help of psyco it can allegedly approach the performance of cPickle despite being written in around three hundred lines of Python. Smile

    It is also compatible with the Pickle interface and protocols, so any investment you have put into making your objects Pickle compatible isn't wasted if you switch.

  • Lawrence Oluyede - Summer of Code

    Lawrence has taken on a google Summer of Code project to port some of the CPython standard library modules to using ctypes. Now that (?) ctypes is supported in PyPy it means that more of the standard library will be available to it. You can read about his progress in his blog.

    The news from the PyPy project in recent months has been getting more and more exciting. It looks like it won't be long before it has practical uses.

  • Python 2.5 Beta 1

    Anthony Baxter has announced the customary Python trunk freeze prior to the release of Python 2.5, beta 1.

    There is also an early version of py2exe available for Python 2.5. Now that I have returned to working on Movable Python I will try and release a Python 2.5 version ASAP.

  • IronPython Beta 8

    IronPython beta 8 is now available. Very Happy

    It has lots of bugfixes that are helpful to us at Resolver Systems and it's good to see the team continue to make steady progress. I've nearly completed another entry in my series on IronPython & Windows Forms by the way. I'm aiming for one a week, and on average I think I'm still ahead of the game. Laughing

  • HLVM

    The High Level Virtual Machine is a platform built on the Low Level Virtual Machine. It aims to provide a common and efficient runtime for implementing dynamic languages; including Python.

    You provide a language definition and production rules for turning source code into an HLVM AST, and it compiles the AST to bytecode [2] (Which the LLVM executes using it's JIT compiler). As well as being an ideal platform to experiment with languages on, it opens up the possibility of language interoperability (since all languages compile to the same AST).

    Unfortunately implementing Python seems to be quite far down the line in their plans. This either reflects the developers personal preferences, or the fact that the dynamic features of Python make it harder to implement.

    Either way I'm very tempted to experiment. I would like to write a compiler sometime, perhaps implementing my Python Extension Language idea. If I do, I will probably target the .NET platform and work my way through this excellent little online book "Let's Build a Compiler for the CLR". The difficult part (even for a simple language) would be static type annotation from source code.

    As well as Mono (as a cross platform target for a dotnet language), there is a GNU project called Portable dotnet that I've only recently come across.

That's enough for today, except perhaps to say that tomorrow I'm off to the Microsoft Office System Developer Conference in London. I'm not sure I'll have anything interesting to report back here though. Hopefully next year I will be able to afford to go to one of the Python conferences.

[1]Unpickling untrusted pickles is a security hole, as I'm sure you're unaware. Unpickling can cause arbitrary code execution.
[2]Or something like that anyway. Wink

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-06-20 12:36:29 | |

Categories: ,

PyZine Goes Free

emoticon:acrobat PyZine has long been an interesting and useful resource for Python programmers. Unfortunately it has languished recently, due to the difficulties the publishers have had in sourcing quality articles [1].

The publishers behind PyZine have their fingers in some other projects (like Open Source Experts) and until their attention returns to PyZine, they have opened up the archives.

So (for the moment at least), you can freely browse the eight published online issues, which have a wealth of material and information.

[1]This is actually more true of the companion Zope Mag, but also applies to PyZine.

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-06-19 11:28:31 | |

Categories: ,


emoticon:dollars I haven't updated the Pythonutils Package for a while. Several of the modules in it have more recent versions available, so it's about time. Smile

As soon as StandOut 3 is ready for release, I will also release an updated Pythonutils package.

I'm going to drop the listquote module, as it is unloved and unused [1].

I'm also considering dropping the pythonutils namespace and making pythonutils a bundle of modules rather than a package. For backwards compatibility in the forthcoming 0.3.0 release I can include a module which imports everything and behaves like the pythonutils package.

[1]Even by me. Laughing Back in the dim and distant past it was needed for ConfigObj.

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-06-18 18:32:19 | |

Categories: ,

StandOut 3.0.0 Alpha

emoticon:avocado I managed to go an entire week without a Python related entry. Of course, despite this lots of things have been going on. I have a stack of potential blog entries metaphorically at my side, some of which may never see the light of day. But first the most important thing.

There is now a new version of StandOut in subversion :

StandOut 3.0.0 Alpha

There are also unit tests to go with it.

StandOut is a simple class that redirects sys.stdout and sys.stderr. This serves two purposes. By passing in a file name when you instantiate StandOut it can log everything printed (and all errors) to a file. You can also assign messages a priority, and tell StandOut what priority of messages to actually display, making implementing different "verbosity levels" very easy.

When I first wrote StandOut I thought I would use the verbosity features a lot. I didn't actually use it for anything except logging until recent changes in rest2web. I was amazed that it worked, and that it was easy to use. Unfortunately along the way, StandOut had grown lots of cruft and complicated ways of using it that nobody needed.

This version is a complete rewrite. The code is better quality, and it has a simpler and straightforward API. It isn't compatible with the previous version, so programs currently using StandOut will need minor modifications to use the new one.

Enough prattle, here's how to use it.

By default StandOut diverts both stdout and stderr. If you pass in a filename (optional) then that will be used to log to. Output on stderr will be prefixed with "[err] " [1].

import os
import sys

from standout3 import StandOut
filename = os.path.join(os.path.expanduser('~'), 'logfile.txt')

standout = StandOut(filename)

print 'hello' # replace with real code ;-)
sys.stderr.write('This is an error')


The log file will then contain :

[err] This is an error

The standout.close() call restores sys.stdout and sys.stderr to their normal state and closes the logfile. If an unhandled exception occurs during your code, execution will terminate. If you want to guarantee that close is still called, you can put it inside a try:... finally: block. Unfortunately the exception will then be raised (by the finally clause) after close has been called. You can use the following trick to get round this :

import os
import sys
import traceback

from standout3 import StandOut
filename = os.path.join(os.path.expanduser('~'), 'logfile.txt')

standout = StandOut(filename)

    raise TypeError("Some Error")

You will see that the error still appears in the logfile in the usual way. On the other hand, if you just let your program terminate (without trapping errors like this), then the open log file will be garbage collected and closed for you. Smile

By default all messages have a priority of five. There are four possible output methods :

  1. The stdout stream (outStream)
  2. Logging output to the file (outLogfile)
  3. The stderr stream (errStream)
  4. Logging errors to the file (errLogfile)

You can configure their threshold independently, but there are easier ways as well. If a message has a equal or higher priority than the threshold for an output method, it will be passed on. If the priority is lower than the threshold, it will be dropped. Easy hey. Razz

The default threshold for the output stream and file is five. The default threshold for the error stream and file is zero (everything logged).

import os
import sys

from standout3 import StandOut
filename = os.path.join(os.path.expanduser('~'), 'logfile.txt')

standout = StandOut(filename, priority=4)

print 'Priority is ', standout.priority
print 'Output message with a priority of four'
print >> sys.stderr, 'Error message with a priority of four'

standout.priority = 6
print 'Output message with a priority of six'
print >> sys.stderr, 'Error message with a priority of six'


Unfortunately the message telling you that the priority is four won't be displayed because the threshold is above the priority. (Threshold 5, priority 4.)

Logged to the file will be :

[err] Error message with a priority of four
Output message with a priority of six
[err] Error message with a priority of six

There is also another way of setting the priority of individual messages, which you may prefer. This is what I used to implement varying levels of verbosity in rest2web.

import os
import sys

from standout3 import StandOut
filename = os.path.join(os.path.expanduser('~'), 'logfile.txt')

standout = StandOut(filename, errThreshold=5)

sys.stdout.write('Message with a priority of six', 6)
sys.stderr.write('Error with a priority of six', 6)

sys.stdout.write('Message with a priority of four', 4)
sys.stderr.write('Error with a priority of four', 4)


Because we explicitly set the threshold for errors to five, only output and errors with a priority of five (or greater) will be passed on. The above code results in the following output :

Message with a priority of six
[err] Error with a priority of six

The following properties (which can also be passed in as keyword arguments) are used to set different thresholds for the different output methods :

  • standout.threshold - get or set the threshold for all of them
  • standout.outThreshold - get or set the threshold for the output stream and output logfile
  • standout.errThreshold - get or set the threshold for the error stream and error logfile
  • standout.outStreamThreshold - get or set the threshold for the output stream
  • standout.outLogfileThreshold - get or set the threshold for the output logfile
  • standout.errStreamThreshold - get or set the threshold for the error stream
  • standout.errLogfileThreshold - get or set the threshold for the error logfile

If the output methods have different thresholds, then standout.threshold (etc) will return -1.

There is currently no way of logging to a separate file for errors, other than instantiating StandOut twice :

import os
import sys

from standout3 import StandOut
filename = os.path.join(os.path.expanduser('~'), 'logfile.txt')
errorFile = os.path.join(os.path.expanduser('~'), 'errorLog.txt')

standout1 = StandOut(filename, stdErr=False)
standout2 = StandOut(errorFile, stdOut=False)

... # code


I developed this using the test driven development methodolgy we use at work. This guides the development process and results in better code and a better API. Someday I may do a blog entry about it. Laughing

The result is an 8k module with 38k of tests. I'm sure the tests could be better factored, but I think the code is pretty good and well tested.

There are still a few questions before I do a final release :

  1. Should I build in a way to log to a separate error file with a single instance of StandOut ?
  2. If you can specify a separate error file, should errPrefix default to '' ?
  3. Should you be able to pass in open files as well as filenames ?

I personally think that using two instances for separate logfiles is fine. This kind of answers questions one and two. I don't see the need for passing around open files, but it would make testing easier.

Anyway, feedback appreciated.

[1]The final module will be called standout rather than standout3. The name is temporary to avoid clashes with the current standout module. You can change the error prefix by passing in a string with the errPrefix keyword. It defaults to "[err] ".

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-06-18 18:07:18 | |

Categories: ,

Hosted by Webfaction