Python Programming, news on the Voidspace Python Projects and all things techie.

Today at Work: Assembly, C/C++, C#, IronPython and Python

emoticon:lighton Today at work has to be the most diverse (in terms of technologies) that I've ever had. I've been working with C/C++, C#, assembly language and both CPython and IronPython. Let me explain...

At Resolver Systems we have an open source project to allow you to use Python C extensions from IronPython: C-Extensions for IronPython. This is not a trivial task by any stretch of the imagination.

One of our developers who has been working on this project spent Christmas in India, and of course the natural thing for him to do was to hack whilst on the beach (without the aid of the web - solving difficult problems without google is possible!). The great news is that he made significant progress.

The basic approach is that we create and load a fake python24.dll [1] (written in managed C/C++). This is initialised with a jump table (using assembly - yay!) that provides the Python C API functions with pointers to managed delegates implemented in C#. Extensions call into our fake DLL, which calls the delegates and create managed objects instead of C Python objects (eventually it will construct 'real' IronPython objects - at the moment it is executing generated code in an IronPython engine). Whilst away, William got enough of this implemented that he could successfully load the zlib module (zlib.pyd) and compress a string. At the start of today it was crashing on decompression. Now it successfully raises an exception (we implemented some of the C API methods for creating and setting exceptions) instead of segfaulting, which is progress I guess. Smile

William has created a nice solution for refcounting, but we don't yet know what we will do about the GIL. Now all that is left is implementing the remaining parts of the C API!

This was extremely interesting as it is the first time I have worked with the Python C API and it was quite a deep dive. My only real experience of C is from assembly language programming on the Amiga, where I had to create C structures in order to make operating system calls (plus I've read a lot). Fortunately I was able to follow most of what William was doing (I was watching and learning rather than contributing), which was cool. William will hopefully write up his work and post the code shortly.

At lunch time I did some more work on ConfigObj, which is straight Python code. I've fixed a couple of bugs in validate. I have a bit more to do and I'll put together a fresh release.

[1]Or .so, there is nothing in this approach that can't work on IronPython with Mono.

Like this post? Digg it or it.

Posted by Fuzzyman on 2008-01-16 23:40:53 | |

Categories: , , , , ,

Resolver One is Now Live!

emoticon:exclaim After two years, two months and two days - Resolver One is now live!

If you're using the latest beta, then you should be informed of this update next time you run Resolver. You can download the free non-commercial version, or pay for the commercial use version of Resolver One. The price will be $199, but as a launch special will be available for $99 for a short while [1].

The new release includes the following changes since beta 7:

  • A couple of bugs relating to exporting formatting to Microsoft Excel format have been fixed
  • The addition of a 'per-workbook' license dialog
  • Final aesthetic tweaks - including new help menu links and the 'clear output' button on the output pane becoming a context menu option
  • Updated samples, documentation and quick-start guide

This is great news, and today has been a lot calmer than yesterday when we were desperately trying to get everything required for this release finished!

For those of you who don't know what I'm talking about, Resolver One is an IronPython spreadsheet/rapid business application platform/IDE. The Resolver Hacks Website explains all about Resolver One!

[1]With all updates to Resolver One 1.X included and support by email and on our support forum.

Like this post? Digg it or it.

Posted by Fuzzyman on 2008-01-16 16:18:55 | |

Categories: , , Tags: , ,

IronPython in Action, ConfigObj and doctests

emoticon:pill I've just finished the only minor edits that were necessary to chapter 9 of IronPython in Action. It was one of the hardest chapters I've written (on WPF), but also needed the least changes after first draft, which was a great relief. Christian Muirhead has also completed the first of his two chapters, this first one on using ASP.NET with IronPython. This marks the two-thirds completion point for the book, and the 'second-third' will shortly go for review and then appear in the early access program. There are some fun chapters coming up, which I'm looking forward to writing (on Silverlight, embedding and extending IronPython and the next chapter on sysadmin tasks with IronPython).

As there wasn't much to do to finish chapter nine, I've taken some time to give ConfigObj some of the attention it deserves. This has meant coding with CPython again, and it has been like returning to an old friend! Once the book is out of the way (something I am really looking forward to), I will be spending more time with this capable companion. Smile

Things that I'm doing to ConfigObj include:

  • Guaranteeing that written config files are newline terminated
  • Quoting keys, values and list members that contain hashes (a bugfix that I'm really surprised no one has reported until now)
  • Review a patch that adds a restore_defaults method
  • Evaluating how difficult adding a reload method would be
  • A couple of bugfixes to the validate module

I have a cache of emails to go through which might turn up a few other issues, but if you have anything other feature requests (or bugfixes) then let me know (preferably on the ConfigObj mailing list). I'll only be able to look at simple tasks though before I return to the book. I'll keep updates in SVN, so in a few days I will be asking people to kick the tyres of the changes.

Working on ConfigObj has also meant a return to doctests. Writing the tests for ConfigObj, with Nicola Larosa, was my first experience of automated testing with Python. Since then I have used unittest extensively at Resolver. Test Driven Development (test first) is now my preferred development technique [1] and I invariably turn to unittest when writing new code.

Writing new tests for ConfigObj with doctest is pretty straightforward, but I still prefer unittest:

  • Testing with doctest fells like I am writing functional tests for my API rather than writing unit tests
  • I lose my IDE support when coding inside a docstring [2] (I lose auto-completion, goto definition, smart indentation, function call tips to name but a few)
  • Having to compare with stdout, which means you are sensitive to things that are sometimes arbitrarily ordered, forces workarounds and limits the way you test (you have to force the behaviour under test to produce output)
  • Writing tests as 'text rather than code' still feels weird (especially all the extra '>>>' and '...' you have to type)
  • Escaping escaped characters for doctest can be a pain - one of my tests now contains "'\\"#a\\"" = \\"b # something\\"\\\\n'"
  • I am much more familiar with things like scoping and setup/tear down with unittest (which is not at all the fault of doctest of course)

Despite this apparent rant, I have nothing against doctest. I do think it is much more suited to creating (and verifying) 'executable examples' in documentation than for unit testing. I find it surprising that some people put so much emphasis on doctest as a strength of Python though, and am also surprised that some people have so many difficulties with unittest which I have always found quite natural (since learning to use it anyway).

[1]If you haven't already seen it, then this is a brilliant article (with pretty pictures!) on why you should test first: Six Reasons to Develop Your Tests First.
[2]And copy and pasting interactive sessions doesn't work so well when doing test first.

Like this post? Digg it or it.

Posted by Fuzzyman on 2008-01-13 00:14:43 | |

Categories: , , Tags: , , ,

Hosted by Webfaction