Python Programming, news on the Voidspace Python Projects and all things techie.

Silverlight and the London .NET Group

emoticon:animals_cat Yesterday I went to a talk on Silverlight by Tim Sneath, a Microsoft 'evangelist'. This was a talk organised by the London .NET User Group [1].

The first part of the talk was about Silverlight 1.0, its video streaming and animation capability. This side of Silverlight doesn't interest me at all, and I guess it is aimed at luring away those currently using flash for adverts. There were some impressive demos though: a video jigsaw that split into seventy jigsaw shaped pieces each showing a fragment of the video, which you then have to reassemble.

The second part was about Silverlight 1.1, which of course can host the DLR and be programmed with IronPython. The single largest factor blocking Silverlight from being used to create rich web applications is the fact that it comes with no controls. (Although at least one chap is having fun making his own.) Thankfully the next two releases, beta late summer and final early 2008, will rectify this with quite a rich set of controls being supplied.

In mitigation (and the important fact) is that Silverlight has full access to the browser DOM and you can make calls from Javascript into silverlight code and from Silverlight into Javascript. This means that you can already write the presentation layer of a client side web app in Javascript (using any of the rich sets of controls that already exist) and implement your business logic in IronPython. This combination of Silverlight and HTML hasn't been talked about much, but is much more interesting to me than messing around with XAML. Smile

Tim talks about the subject in his blog entry Programming HTML with C# and he recommends a MIX07 Session Talk by Stefan Shackow on the subject. There is also a quickstart on calling into Silverlight from Javascript. At some point I will have a play and see if I can come up with an interesting demo for IronPython, Silverlight and HTML / Javascript.

[1]A marginally more sober group than the London Python Crowd, but only just. Laughing

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2007-06-15 23:48:42 | |

Categories: ,


Windows Flying Dudes: The Agent Server Objects

emoticon:target Have you ever seen this dude before?

A strange Windows wizard dude

Apparently he is called Merlin, and he also does this:

Merlin doing odd things

He is part of a bizarre feature of Windows called 'AgentServerObjects'. These are little animated characters that fly around the screen, making announcements and doing quite odd and quirky things. They don't get used very much, in fact it is hard to see what you could use them for. But they're certainly fun.

In order to run this example I generated the interop dll in 'c:\'. This uses tlbimp, which comes with the .NET framework SDK.

C:\>set PATH=%PATH%;C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin
C:\>tlbimp c:\WINDOWS\msagent\agentsvr.exe

Some IronPython code to make this dude fly around your screen:

import sys
sys.path.append('C:\\') # or wherever your interop dll is

import clr
clr.AddReference("AgentServerObjects.dll")
from AgentServerObjects import AgentServerClass

a = AgentServerClass()
id, rest = a.Load("merlin.acs")
ch = a.GetCharacter(id)

ch.SetSize(128,128)
ch.MoveTo(0, 0, 0)
ch.Show(0)
ch.MoveTo(600, 100, 2000)
ch.Play("Announce")
ch.MoveTo(300, 150, 1000)
ch.Play("Think")
ch.Play("Search")
ch.Play("Congratulate")
ch.Hide(0)

Oddly cool... Smile More details here.

I've written this up in the IronPython Cookbook. Davy Mitchell has also added an example of Playing MP3s.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2007-06-15 22:56:03 | |

Categories: ,


Gnuplot Frustration

emoticon:eyeballz My Dad does computer related safety and reliability consultancy. One of his recent projects has been calculating blast danger zones for power stations - how far out from a power station do you need to build if you want the buildings to be safe from flying concrete, in the event of an explosion! This means plotting 'risk contours' for the area around a potential blast zone. He has been using a script to generate the contour images, and frankly the images it produces are rubbish. Smile

As Resolver can process this kind of data, I thought it would be fun to hook it up and see if I can generate some charts for him. He would really like some vector images, to embed into his drawing package.

We use Gnuplot at work to generate some codemetric charts I thought I would start with that. The images it creates are not very glitzy, but it can produce an enormous range of different types of charts (except pie charts).

Trying to get it to work on Windoze was a tale of pain and sorrow I'm afraid. The basic problem is that the Windoze build has no error reporting of any kind. It doesn't even complain when you ask it to build a non-existent script!

When I had got to grips with the basics of the scripting language I managed to coerce gnuplot into producing an empty png file, of course with no hints as to what the problem was. What the README.Windows file doesn't make clear is that the two environment variables for the path to the fonts directory (which has been the same on every Windows machine I've ever used) are essential. Once I'd set those, I eventually got an image out.

  • GDFONTPATH: the directory where png terminal searches for TrueType fonts.

    SET GDFONTPATH=C:/WINDOWS/FONTS

  • GNUPLOT_FONTPATH: used by the postscript driver as search path for TrueType fonts.

    SET GDFONTPATH=C:/WINDOWS/FONTS

I generated a test data csv file (test.dat) containing dates and random numbers:

25/07/2007  -99.759
26/07/2007  -62.111
27/07/2007  -33.785
28/07/2007  -20.832
29/07/2007  -27.133
30/07/2007  4.331
31/07/2007  22.647
01/08/2007  54.885
02/08/2007  80.198
03/08/2007  88.198
...

A gnuplot script to generate a chart from this is:

set timefmt "%d/%m/%y"

set term png
set output "test.png"
set title "Random Numbers Against Date"
set style data lines
set key left
set grid

set xdata time
set format x "%d/%m/%y"

set ylabel "A Random Value"
set xlabel "The Date"

plot 'test.dat' using 1:2

Which produces:

A sample gnuplot image

That was far more effort than it should have been (and I don't really fancy learning another language just to generate charts that aren't as good as I'd like). I'm doing this from IronPython, so I'm launching it as a subprocess anyway. I might as well launch a Python script using matplotlib. For my Dad's machine I can wrap it with py2exe. I know that works because matplotlib plays fine with Movable Python.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2007-06-15 22:43:25 | |

Categories: , , ,


Listing Processes with PyWin32

emoticon:exclaim A few days ago I posted about how much easier it was to list all the running processes with IronPython than the pywin32 code we were using.

Well of course we weren't using the best techniques with pywin32 either, and Tim Golden has suggested a couple of better solutions:

WMI using pywin32:

import win32com.client
wmi = win32com.client.GetObject ("winmgmts:")
procs = wmi.InstancesOf ("Win32_Process")
print "\n".join (proc.Properties_ ("Caption").Value for proc in procs)

Not quite as nice as the IronPython example. Luckily Tim has created a wrapper around pywin32 for WMI stuff.

WMI using wmi module:

import wmi
procs = wmi.WMI ().Win32_Process ()
print "All running processes:"
print "\n".join (proc.Caption for proc in procs)

Very nice, but has an additional dependency.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2007-06-15 15:25:07 | |

Categories: ,


PyCon Italia and London Python at the IET

emoticon:acrobat Well, PyCon Italia has just finished. I wasn't there, but Nicola Larosa tells me that it was fantastic. Apparently 200 people attended, which for a national European conference is a lot. It sounds like Python is really making strides in Italy.

Nicola was occupied with organising but particularly enjoyed Alex Martelli and the talk on the OLPC. Come on guys (yes Lawrence I mean you), where are the reports?

So we are speeding towards PyCon UK, in September, let's hope it is as successful. Resolver Systems would like to help sponsor this event, but the organisers haven't yet responded... Razz

Meanwhile, there is more going on in the UK. Specifically, London Python event at the IET's Savoy Place building:

Pete Ryland, Tim Golden and Michael Glazebrook are putting on an evening Python event at the IET (Institute of Engineering Technology) at Savoy Place on 5th July.

Although the talk bit is not aimed at code-gods such as yourselves, that's not really the point ! Though we hope it'll be interesting even so. This is a networking event (free grub & booze) with more time spent socialising than being lectured to. If we can prove enough interest we can hope to get the use of the facilities for as many talks and lectures as we'd care to present. So an important part of this is for us to hear from you what you'd like (and if you're prepared to give a talk).

http://www.theiet.org/

The event is not yet on the calendar (should go up this week), so here's the description:

Date & Time:18:30 – 20:00 5th July 2007 Networking and Wine reception 20:00 - 22:00
Event:A light byte of Python
Venue:The IET, Savoy Place, London, WC2R 0BL
Cost:Free
Talk by:Michael Grazebrook, Tim Golden and Pete Ryland

Python is a modern language. Easy enough for the engineer seeking to dabble in simple programming, but powerful enough to be used for serious development (I learned it working for banks). It is also free: open source.

This is an event in three parts. We will introduce Python for beginners, showing simple pieces of code which you can modify at home. We’d like to provide a series of lectures, if you prove there is enough demand, so help us debate what you want. Finally we’ll have some PCs available so you can try it out. We hope you’ll go home feeling confident enough to try it out.

The talk will consist of a handful of brief presentations of simple, working programs, around 10 lines long, doing powerful things such as:

  • Opening a Web page and extracting some information
  • Simple user interface design
  • Manipulating data in an Excel spreadsheet
  • Controlling external hardware (an off-the-shelf USB experiment interface board)

If you, our public, show enough enthusiasm we’d like to present a series of evening workshops on Python at the IET. So after the main talk we’ll have some debate about what you would like. The event is supported by the London Python Users Group, who are looking for somewhere a bit more practical than a pub to exchange experience.

If you bring a lap-top, we will help you get the examples running after the talk – even if you have never used Python before.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2007-06-11 21:41:44 | |

Categories:


Resolver, IronPython and Unicode

emoticon:avocado At work today I was working on a bug with importing data from an external data-set. This turned out to be a bug in our handling of non-ASCII strings, compounded by a bug in our error handling.

In the space of two hours, Jonathan and I went through the following emotions:

  • How do we fix this bug?
  • Ah, I think we can see what is going on.
  • Aargh, serious unicode bug - Resolver is totally screwed!
  • Oh. Here's the two line fix that sorts it completely.

To be fair it was actually four lines. The two lines needed to go in two places.

The bug came about at least partly because the handling of Unicode is a bit different between CPython and IronPython (and also because we knew we had a Unicode related bug which this has forced us to address).

In IronPython, the str type and the unicode type are the same object (the .NET string type is Unicode). Here are a couple of examples of the differences:

CPython 2.4.4:

>>> s = '\xa3'
>>> s + u'33.33'
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode
byte 0xa3 in position 0: ordinal not in range(128)

Adding a byte string to a Unicode string, resulting in an implicit decode which fails.

IronPython 1.1:

>>> s = '\xa3'
>>> s + u'33.33'
u'\xa333.33'

Both strings are already Unicode, no decode needed. This can make Unicode handling a lot easier with IronPython.

Second example.

CPython 2.4.4:

>>> s = u'\xa3'
>>> unicode(s)
u'\xa333.33'

Calling unicode on a Unicode string is effectively a no-op.

IronPython 1.1:

>>> s = u'\xa3'
>>> unicode(s)
Traceback (most recent call last):
  File , line 0, in <stdin>##5
UnicodeDecodeError: Unable to translate bytes [A3]
at index 0 from specified code page to Unicode.

Here the unicode call, is the same as calling str - which does result in an implicit decode which fails.

Our problem was essentially caused by calling str on objects which were already strings. As soon as we stopped doing this, the problem went away [1].

Another important result of all this, is that you should not use the (Iron)Python string for storing binary data. Instead you should use byte arrays. This means that things like zlib compression and the binary pickling protocol with strings don't always behave correctly. (They don't always misbehave either, you just need to be very careful. I'm not sure how much IronPython magic is going on with these types.)

Loading data into a byte array with IronPython is a bit more verbose than reading a file with Python, but not so bad:

>>> from System import Byte, Array
>>> from System.IO import FileStream, FileMode
>>> s = FileStream(filePath, FileMode.Open)
>>> bytes = Array.CreateInstance(Byte, s.Length)
>>> s.Read(bytes, 0, s.Length)
9111
>>> s.Close()

The mutable bytes array (bytes) now contains the data read from the file. The number returned by s.Read(... is the number of bytes read.

update

Max suggests some slightly shorter code for reading byte arrays from files:

>>> from System.IO import File
>>> bytes = File.ReadAllBytes(filePath)

Smile

[1]Well, so far anyway. Wink

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2007-06-11 21:02:19 | |

Categories: , , ,


IronPython is Better than pywin32 for Some Tasks

emoticon:torch I'm sure this will come as no surprise, for some Windows specific tasks IronPython is a much better fit than pywin32.

For example, certain background processes can bork a build. These are notably Thunderbird and the AVG anti-virus system scan. Both of these suck up CPU cycles in the background and interfere with the build, particularly our performance tests.

We have a simple script bad_processes.py, which checks that these processes aren't running and aborts the build if they are. This currently runs under CPython and uses pywin32.

Recently this has started failing on one machine, with an obscure error. All google has to say on the subject, is that the cause of the error might be a non-English Windows version or a very old pywin32 install. Neither of these is the case.

Fortunately the IronPython equivalent is very simple. Here is how you get a list of all the currently running processes:

from System.Diagnostics import Process

procs = Process.GetProcesses()
print "All running processes:"
print '\n'.join(proc.ProcessName for proc in procs)

This has the advantages of being shorter, easier to read and less error prone than our current solution.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2007-06-09 20:28:42 | |

Categories: , , ,


Trying to Read Shortcut Targets on Windows

emoticon:exclaim Now that Resolver is in the hands of beta-testers, documentation is becoming more important. We have a quick-start guide written, but the API to our class library is vital to getting the best from Resolver.

Back at Christmas time I wrote the core of a documentation builder, which would put together module documentation from ReST formatted docstrings, including building an index page. We have finally got round to writing the docstrings and integrating building the documentation into our installer.

It aborts (by throwing an exception), if any members of our public API don't have docstrings or the formatting of any of the resulting pages are invalid. The documentation builder is a core of around 400 lines of Python code, which is nice and easy to modify or extend. Once Resolver has gone public (in July), there is a good chance I will be allowed to release this.

The final step in integrating the documentation was writing a functional test that runs the installer, checks that a link to the library documentation appears in the start menu and that this link points to a real file in a directory of HTML files. We already have an installer test, so all we needed to do was check the link exists (a '.lnk' file in a specific directory) and check the other files in the directory that the link points to.

So how exactly do you check the target of a Windows shortcut file? This seemingly simple task ended up driving Christian and I around in circles and taking half a day. Surprised

Links files have a binary format (documented here (PDF)), which we didn't fancy parsing. After searching we couldn't find a command line tool or .NET API for reading or following them. Nor could we find more than a couple of people asking about this - it seems odd that no-one else ever needs to do this?

Eventually, through a C# example, we found the Windows Scripting Host (warning: dumb IE only website) which has an API (via interoperability) for accessing shortcuts.

After a while we discovered that you can only use this API to create shortcuts, not to access existing ones! Anyway, I wrote up what we had learned:

Next we discovered a WMI solution, using a very bizarre SQL like syntax for querying the system about shortcut files!

Set objWMIService = GetObject("winmgmts:\\.\root\cimv2")
Set query = "Select * From Win32_ShortcutFile WHERE" &
            " Target = 'N:\\tandem.exe' AND Path = '\\nectem\\'"
Set colFiles = objWMIService.ExecQuery(query)

For Each objFile in colFiles
    Wscript.Echo "Name: " & objFile.FileName
    Wscript.Echo "Shortcut target: " & objFile.Target
    Wscript.Echo "File name: " & objFile.Description
Next

This is more weirdness than we wanted, so we carried on the hunt. In the end I found what we were looking for in the google-cache of an RSS feed from someone's blog!

The final solution is actually fairly easy, using Shell32 interoperability, it just took us half a day to get there! I wrote it up in:

Other recent additions to the IronPython Cookbook include:

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2007-06-09 15:58:31 | |

Categories: , , ,


Hosted by Webfaction

Counter...