Python Programming, news on the Voidspace Python Projects and all things techie.

Resilience Against Errors in User Code: Catching String Exceptions

emoticon:home Suppose you have an application that executes user code. You want to be resilient against errors in that code, you don't want to crash if exceptions are raised in user code, but you want to be able to report the errors.

You might think that the following code would be a good start:

import traceback

except Exception:
    message = traceback.format_exc()

You can find plenty of Python experts recommending that where you want to catch all possible exceptions you should explicitly catch Exception rather than using a bare except clause. Here is one example (of many) with Guido agreeing back in 2005.

However, what happens if the user raises a string exception:

Python 2.5.1 (r251:54863, Jan 17 2008, 19:35:17)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
...   raise 'freshfoo'
... except Exception:
...   print 'Caught!'
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>

The exception handling block does not catch the string exception.... ouch...

A bare except does catch it, and you can then fetch information about the exception using sys.exc_info():

>>> try:
...   raise 'freshfoo'
... except:
...   print sys.exc_info()
('freshfoo', None, <traceback object at 0x...>)
>>> try:
...   raise Exception('blam!')
... except:
...   print sys.exc_info()
(<type 'exceptions.Exception'>, Exception('blam!',), <traceback object at 0x...>)

sys.exc_info() returns a tuple. For 'normal' exceptions the first value is the exception type. For string exceptions it is the string itself. The second value is the exception instance, or None for string exceptions. The third member is the traceback object.

An interesting snippet from the Differences Between IronPython 1 and Python 2.4 document mentions that IronPython catches string exceptions by value (strings are value type objects in .NET - allocated on the stack rather than the heap) whereas CPython catches them by reference. This means that the following works in IronPython but not in CPython:

    raise 'abc'
except 'a' + 'bc':

String exceptions (and raising non-exception objects like instances of old style classes) have been deprecated for some time [1] of course - exactly because of the problems described in this article. They raise deprecation warnings in Python 2.5 (I cut them from the examples here) and are gone altogether in Python 3.0 (see PEP-0352 Required Superclass for Exceptions). However, if you are executing arbitrary user code with a current version of Python then you may still need to be aware of them. Surprised

By the way... if you are executing code from strings or compiled code objects, then you may want to consider not using functions directly from the traceback module. It uses linecache, which can potentially make several disk accesses (including one per path in sys.path [2]) for every frame in the traceback trying to find the original sources.

It is easy enough to implement a custom traceback formatter by copying and modifying the extract_tb function from traceback. It takes a traceback object which you can retrieve with sys.exc_info().

[1]String exceptions are a very old feature of Python. The move away from them started as long ago as Python 1.5.2. Until recently they could still be found in very old codebases like mailman though.
[2]Compiling code objects with a name surrounded by angle brackets short circuits this.

Like this post? Digg it or it.

Posted by Fuzzyman on 2008-07-18 23:04:15 | |

Categories: , Tags: , , ,

Blog on Blogging: Inside the Blogosphere Questionnaire

emoticon:test_tubes Interesting that this should come up at the same time as the last entry. I recently got asked to take part in the Inside the Blogosphere: Blogger Questionnaire. Not to waste the answers, and being a while since I've blogged about blogging - here they are...

  • Please enter your blog(s) url with a short description below:

    • The Techie Blog

      My technical blog that documents the Python programming world, my work at Resolver Systems where we are developing a revolutionary new kind of spreadsheet and my writing activities.

    • IronPython URLs

      Keeping track of new and interesting things happening in the world of IronPython and the Dynamic Language Runtime.

    • My Personal Blog

      My personal (and oldest) blog, with an archive of my original blog started over five years ago. In the past year I have been busy writing a book (IronPython in Action) and so this blog has been neglected with a backlog of posts I need to find time for.

  • Why did you start blogging?

It seemed an interesting creative outlet. I'm opinionated, so blogging was a natural platform to express my opinions and rants.

As I learned to program it also seemed a great way to document my learning for others and publicise the tools and libraries that I was creating. (Even though the early ones weren't much good.)

I first started blogging in early 2003, as I was renewing my interests in computers. My first blog started at the same time as I started to develop my own website (remnants of which remain - not good!). As I learned to program I developed a writing style and online persona - and most of my blogging became technical.

I also gained an audience, which is a great motivator. The Python (programming) community is very friendly and I was surprised by how ready people were to read what I wrote! Blogging without readers is pointless.

  • What motivates you to blog?

I have become a well known member of the Python community almost entirely due to my blog. I want to promote the Python programming language, and I enjoy writing and teaching.

Individual posts may have very different motivations. It could be something I think is important to the communities I'm part of, something I want to promote to those communities, something I've done or simply something that I find interesting. It's also nice to be able to post news before anyone else does.

Some posts (often the more controversial or interesting ones) are simply me exploring a subject and trying to express something about it.

  • What is your long term goal of your blog(s)?

I see my blog as a way of teaching and promoting the Python programming language, something I feel passionately about. I would like to develop my writing skills and explore issues that interest me. Particularly software quality, usability, agile development processes, testing techniques and related topics. Some of these are things I haven't really begun to explore yet in my writing.

In the long run I would like to be a freelance programmer and writer, and I definitely see the blog as a big part of achieving that goal.

I also want to make a difference in the software community - furthering my craft. My blog is an excellent platform for communicating and exploring those issues.

I'm also a member of several different 'communities' around programming. A lot of the conversation around those communities happens on blogs and I want to strengthen this.

Towards all those goals I would like to write longer articles. It is sometimes hard to see how those fit in as blog posts though, as they become articles in their own right. In a way I don't draw much distinction between writing on my blog and developing my website - although the style of writing can be very different. Blogging can be a very 'free-form' way of writing which is refreshing.

  • What is the biggest mistake that advertisers make in the blogosphere?

Not making their adverts relevant and interesting. Advertising is easy to screen out - it needs to be engaging and part of the 'content'.

  • If you could start over what changes would you make in regards to blogging?

I'm not sure I would make changes. Blogging is a personal and creative outlet for me and I feel free to write what I want.

I would probably have maintained my personal blog a bit better during the last year, despite being bogged down in a writing project.

Like this post? Digg it or it.

Posted by Fuzzyman on 2008-07-16 14:33:21 | |

Categories: , , ,

How Did I Get Started In Software Development?

emoticon:beaker Scott Cowan, of the appropriately named Sleep Overrated blog, has tagged me, asking how I got started in software development. I say appropriately named, because it's now 3am and I'm facing a second night without sleep. sigh

Oh well, at least it means I have time to answer this.

  • How old were you when you first started in programming?

Like Scott I started programming when I was very young. My father brought home a BBC Micro Model B (32kb RAM with a 2MHz 6502 processor) when I was around 8 or so. It wasn't long before I started programming with the builtin language BBC Basic which was a sophisticated version of Basic allowing structured programming with named procedures (at a time when MS basic was limited to effectively two character variable names).

After the BBC I moved onto the Amiga, a machine of beauty and elegance. I started with a dialect of basic called Amos, from which I learned to make operating system calls. The Amiga operating system was a joy to program with and I moved on to Motorola 68000 assembly language programming, building up the C structures required to make OS calls.

This occupied me until I went to university, where I started a law degree. I completed the first year, but dropped out in the second due to personal difficulties. I eventually settled down with a job in an independent builders merchant where I stayed for ten years becoming sales manager at one of the branches.

In early 2003 my interest in computers was rekindled and I started learning HTML and blogging.

  • What was the first real program you wrote?

I started programming again as a hobby. A bunch of us were playing an online email game called Atlantis [1]. We figured that we could write a small program that consolidated our reports and helped us make decisions. At the time none of us had programmed for a while, and the only computer I had at the time (except for at work) was a PocketPC device.

So the only constraint on choosing a programming language was that it had a version that ran on the PocketPC. We had decided on Squeak when at the last minute someone suggested Python.

We started the project which we called Atlantibots. One of the guys (Rob McNuir I think he was called) created a parser that read in Atlantis reports and built a datastructure. I built various things on top of this, including maintaining a full map of all the explored area, and consolidating group reports.

My proudest part of the project was a simple implementation of Dijkstra's shortest path algorithm that would generate movement orders for your units - calculating the quickest route to a destination.

The Atlantibots program is still available and still gets downloaded a few times a day. I loved programming in Python, and spent more time on that as my interest in Atlantis faded.

  • What languages have you used since you started programming?

I've already mentioned Basic and 68k assembly language.

The language that I have overwhelmingly the most experience in is Python of course.

I've also dabbled in Javascript at various times. It's slightly odd, but if you ignore the indignities that browser's force upon you, it is quite a nice dynamic programming language.

In the last two years of working with IronPython on the .NET framework I've used C# quite a bit. I've had to read it a lot more than I've written it, but in writing the examples for IronPython in Action I've ended up writing quite a bit of C# and VB.NET. I quite like C#. It's no Python, but it is generally unsurprising and easy to learn.

  • What was your first professional programming gig?

After programming in Python for a couple of years as a hobby, I decided I'd rather be programming than selling bricks. I expected to have to get a code monkey job programming in PHP or some other abomination for a couple of years in order to build up a career.

As it happened I was lucky. Resolver Systems were hiring, having started work on a .NET spreadsheet development application in around November 2005. They had chosen IronPython as the development language because as well as being suited to the application it fitted very well with the agile development processes they were using.

I was the fourth developer to join the team (in June 2006), and the first one with any experience of Python.

  • If you knew then what you know now, would you have started programming?

Definitely. I really enjoy programming. I would probably have done a computer science degree had I known I enjoyed it so much, but then I probably wouldn't be programming in Python now.

  • If there is one thing you learned along the way that you would tell new developers, what would it be?

How much of a difference Test Driven Development made to the way I think about and approach programming. I can't imagine developing anything of any complexity without an automated test framework, and I find TDD to be an ideal way to handle the micro-design of any project.

It takes a bit of discipline, but it pays off so quickly.

  • What's the most fun you've ever had programming?

I'm still really enjoying working at Resolver Systems. I think Resolver One is a great product and I have some really good colleagues.

[1]Anthony Briggs maintains a Python version of Atlantis called PyBEM.

Like this post? Digg it or it.

Posted by Fuzzyman on 2008-07-16 14:09:13 | |

Categories: , , Tags: ,

Hosted by Webfaction