Python Programming, news on the Voidspace Python Projects and all things techie.

Python and Threading

emoticon:paper In my Python programming so far I've managed to avoid threads altogether.

I learned a lot whilst working on ConfigObj with Nicola Larosa. He has a pathological hatred of threads, which I inherited by proxy.

In order to justify a prejudice like this you need to understand the issues. In his case the loathing almost certainly came from great pain in debugging thread related problems. I understand what problems threading could cause, but had no direct experience.

Working with IronPython and Windows Forms I've had to wrestle with threads a bit. In our production code Timers and Network Clients involve asynchronous callbacks which use threads. More to the point; in order to interact with our GUI a lot of the tests need to run on another thread.

We've had much fun working out timing and blocking issues. We're currently going through our test codebase and trying to remove as many 'voodoo sleeps' [1] as possible and actually resolve the issues they are attempting to work round.

Programming in IronPython and Windows Forms we're using a native GUI framework. My guess is that other Python GUI toolkits also use threads 'under the hood' for timer classes and the like. Because these frameworks are non-native, the threading doesn't normally interact with your Python code. Perhaps this a good thing. Smile

In the last couple of days, working with the boss on a CPython script, I've used the Python threading API for the first time.

The basics are very easy, but there are a couple of noticeable oddnesses.

from threading import Thread
import time

TIMEOUT = 110
def LongRunning():
    time.sleep(100)
    print 'Finished'

thread = Thread(target=LongRunning)
thread.start()
#
# Thread is now running
thread.join(TIMEOUT)
if thread.isAlive():
    print "Oh dear - the thread didn't terminate normally."
    print 'We timed out instead.'
else:
    print 'Terminated normally.'

One slightly odd thing is that the first argument to the Thread constructor is reserved, so you can't use it. Huh ?

Secondly, there is no way to terminate a thread. My boss was most perplexed by this (he is used to threading APIs from other languages which do allow this). If I understand correctly (which is possible - but perhaps not likely), the reason for this is "it could leave your objects in an inconsistent state to terminate at a random point, so you shouldn't do it".

This does seem at odds with the normal Python philosophy of not telling the developer how he ought to do things.

In our case we were trying to test a long running loop by spinning it off on another thread. We got round it by monkey patching one of the functions the loop used to raise an exception. We redirect standard error around the exception to reduce the noise. Not ideal by any means...

[1]Thread.Sleep calls that wait a random amount of time to allow an event to happen, or some other magic to take place, before moving on to the next part of the test. We're replacing them with event waiter patterns etc.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-11-08 23:43:33 | |

Categories: ,


Python Jobs on the HiddenNetwork

emoticon:home Big news: there are now Python jobs on the Hidden Network Jobs Board.

If you're searching for a Python job, or are looking to hire, this is the place. Smile

This is great news. It means that Python jobs will be shown all across the Hidden Network. It's only just happened, so there are only a couple so far, but expect this to grow.

The HiddenNetwork is the job board started by the team responsible for The Daily WTF. Adverts are shown on a network of top programming blogs. That means that the posts get shown many thousands of times a day, and are read by top programmers from all around the world.

So is advertising on blogs an effective way of hiring ?

Well... a few months back I mentioned on this blog that we were hiring at Resolver Systems. We had an excellent developer, Christian Muirhead [1], who applied for the job. He's an excellent programmer. Smile

We've got an interesting mix of skills now, and have been very lucky with hires. With a small team it's very important that everyone fits in. I feel very lucky to work at Resolver, as they're all very good blokes. The team includes :

  • Andrzej with web development experience in Java and Ruby
  • Giles (the boss) who has done a lot of Java development for a large investment bank
  • William, who has done C++ (and Objective C etc) programming (including working on a win32 compatibility layer for porting games to the Mac)
  • Jonathan with a lot of GIS experience in C, C++, C#
  • Christian a web developer with ASP.NET and Python
  • Me Smile

(My apologies to my esteemed colleagues if I have misrepresented them.)

As you can see, we're a diverse bunch. This makes for great pair programming as we've all learned different patterns and idioms and all have different experiences to draw from.

Anyway, I've wandered further away from the point than usual. The theory is that techies who read blogs are more likely to have a passion for programming. It's not just Python jobs, they also have Java, Ruby, .NET, PHP jobs and all the usual suspects. So if you're looking to hire someone who's a cut above the average then the Hidden Network could be what you're looking for. Watch out though, you might get someone like you, or even worse; someone like me. Razz

Oh, by the way: I'm off to Italy for a few days. You'll have to cope without me until next week. Wink

[1]Although he does have a slightly odd sense of humour. If you live in London, check out his Movie Maps application.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-11-08 23:07:44 | |

Categories: , , ,


Movable Python 2.0.0 Beta 2

emoticon:movpy2 I'm pleased to announce the release of Movable Python 2.0.0 Beta 2.

There are now updated versions for Python 2.2 - 2.5, the Mega-Pack [1], and the free trial version.

The most important changes in this release are :

  • Fixes for Pythonwin. The 'run' button and 'grep' dialog now work.
  • You can now specify a directory for config files.
  • The environment programs are run in is no longer pre-populated with variables. The Movable Python variables are now in a module called movpy.

You can see the full changelog below.

You can download the updated version from :

Movable Python Download Pages

You can get the free trial version, from

Movable Python Demo & Other Files

What is Movable Python

Movable Python is a distribution of Python, for Windows, that can run without being installed. It means you can carry a full development environment round on a USB stick.

It is also useful for testing programs with a 'clean Python install', and testing programs with multiple versions of Python.

The GUI program launcher makes it a useful programmers tool, including features like :

  • Log the output of all Python scripts run
  • Enable Psyco for all scripts
  • Quick Launch buttons for commonly used programs
  • Configure multiple interpreters for use from a single interface

It comes with the Pythonwin IDE.

Plus many other features and bundled libraries. Smile

What's New ?

The changes in version 2.0.0 Beta 2 include :

(Many thanks to Schipo and Patrick Vrijlandt for bug reports, fixes and suggestions.)

Updated to Python 2.4.4 and wxPython 2.7.1

Fixed the bug with pylab support.

Fixed problem with global name scope in the interactive interpreter.

Everything moved out of the default namespace and moved into a module called 'movpy'. This contains :

  • filename = the path to the script we are running (the executable in interactive mode)

  • filedir = the directory that script is in (None when in interactive mode)

  • movpydir = the directory of the movpy executable

  • curdir = the cwd from which we have been called

  • configdir = the directory where config files are accessed from

  • libdir = the 'lib' directory that modules/packages are contained in

  • commandline != '' if '-c' was used

  • go_interactive = True if '-i' was set.

  • interactive = True if we are in an interactive session

  • interactive_mode is a function to enter interactive mode

    interactive_mode(localvars=None, globalvars=None, IPOFF=False, argv=None)

  • movpyw = True if we are running under movpyw rather than movpy

The docs menu option will now launch the local version if available.

Logfile is now closed using 'atexit' (should be closed after other 'atexit' functions finish).

Logfile is flushed after every write to keep it in sync.

A new command line option to specify the config file directory.

movpyw --config ~\movpyfiles\config

(The '~' is expanded to mean the user directory.)

Fix for grep in Pythonwin IDE.

Fix so that 'run' works in Pythonwin. (Browse button is disabled.)

Hopefully the next release will be 2.0.0 final, with a few minor changes and completed documentation Exclamation

[1]The Mega-Pack is all the interpreters bundled together. It comes pre-configured so that you can test programs with any version of Python (including IronPython) from a single interface.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-11-07 23:45:57 | |

Categories: ,


BlogAds Gadget Network

emoticon:pen_book Voidspace is now part of the BlogAds Gadget Network, which is very cool. Smile

BlogAds is an advertising network of specialist blogs. The Gadget Network features those with a focus on gadgets and techie subjects.

The adverts are pretty good, you can see them on the left sidebar. They can include an image and a good amount of texts.

There are certainly some higher traffic (and more expensive) blogs than mine, but according to the metrics on the network; an advert on Voidspace will get around 22 000 impressions in a week and cost you $10. Smile

The adverts are site-wide, but they will mainly be seen by alpha geeks like you.

If you have any technical products to advertise, then the Gadget Network is a great place to do it, and especially Voidspace. Wink

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-11-07 23:27:47 | |

Categories: ,


Movable Python Extras

emoticon:pencil There are now some Movable Python extras available for download.

These can be downloaded by anyone, but are especially useful for use with Movable Python. You can find them at :

Python files at Tradebit

The files available are :

Instructions on how to use SPE and matplotlib with Movable Python are included on the details pages for the files.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-11-07 18:24:39 | |

Categories: ,


New Hard Drive

emoticon:drive I've just bought (or at least paid for) a new hard drive.

The most cost effective size seemed to be 320 gigabytes. The two cheapest places I could find (for branded drives) were Dabs and Ebuyer. They were almost identically priced, but the service from Dabs has been better in the past.

The UK cost was the equivalent of about 40c per giga-byte.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-11-04 14:59:45 | |

Categories:


Python import and __main__ Oddness

emoticon:mobile_world Try creating a small Python script with the following code, and running it :

import sys
import imp
x = imp.new_module('__main__')
sys.modules['__main__'] = x
print x

What would you expect to see output ? In theory I suppose it ought to be a module object.

What happens (with Python 2.4.4) is that it prints None.

The reason for this is that when you replace sys.modules['__main__'] with another one, the reference count of the original __main__ module drops to zero. So garbage collection kicks in. The original of course is the script currently running.

Except shouldn't that result in a NameError. I guess the module is only partially garbage collected, hence the name isn't lost but its value is reset to None.

Hmmm... As the module is being executed it shouldn't be garbage collected at all, the interpreter should keep a reference to it [1]. And the moral of this story, don't do this at home. Wink

Note

Python behaves differently if you try this in an interactive session. To see the same results as above you have to run this as a script.

Chris Siebenmann has written up an explanation.

[1]And yes, this did bite me. The answer is to keep an extra reference yourself.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-11-04 10:55:45 | |

Categories: ,


Hosted by Webfaction

Counter...