Python Programming, news on the Voidspace Python Projects and all things techie.

Linux is Great, I Think

emoticon:nightmare From 6pm Monday until around 9pm yesterday (Tuesday) my server was mainly down. This meant no email and no website.

It is a virtual server provided by Unixshell.com, with the virtualisation done by Xen.

I could restart the server, but it would only last a few minutes before wedging itself thoroughly again.

When I logged in via SSH, I got obscure messages streaming down the screen :

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel: Oops: 0000 [#1]

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel: SMP

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel: CPU:    0

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel: EIP is at _mmx_memcpy+0x92/0x125

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel: eax: 00000000   ebx: 03ffe240   ecx: c25ecc94   edx:
c6a9e000

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel: esi: c6b05fc4   edi: c2c681cc   ebp: ffff0c7c   esp:
c252b68c

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel: ds: 007b   es: 007b   ss: 0069

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel: Process exim4 (pid: 7027, threadinfo=c252a000
task=cb8a9030)

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel: Stack: <0>c2c0058c c6a9e384 ffff0c7c 00000138 c019984f
c2c0058c c6a9e384 ffff0c7c

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel:        ffff0c7c c298cd54 c252b710 c2bf1180 c25ecc94
00000010 00000070 00000010

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel:        0000f58c 00000208 c2bf1000 ffffff38 0000000d
c298cd54 00000000 c252b710

...


Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel:  [<c015a426>] __bread+0x6a/0x7e

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel:  [<c01a21dd>] reiserfs_prepare_for_journal+0x63/0x6a

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel:  [<c0193920>] clear_all_dirty_bits+0x9/0xd

Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ...
voidspace kernel:  [<c019eb68>] reiserfs_delete_item+0xf4/0x32a

I missed a vital clue buried herewithin and tried all sorts of things. Updates didn't fix it, top showed no rogue processes, chkrootkit came up clean, memory, disk and CPU usage were all normal. I finally spoke to a knowledgeable friend who suggested it wass likely to be a hardware problem, so I tried memtest (which didn't reveal anything) and raised a support ticket.

In the meantime I decided to clear up some stray packages which I'd installed and wasn't really using, especially MySQL which was originally installed for someone else. I spent a while trying to persuade apt-get to list all installed packages, before discovering that the required magic incantation is dpkg -l. Smile

I could see various packages related to MySQL installed, including ones like mysql-common. This sounded like a good place to start, and without properly reading the messages I started the remove. To my horror I watched apt-get faithfully obey my instructions and proceed to uninstall Exim (my mail transfer agent), lighttpd (my webserver), courier (my POP mail server) and more. All of these depended on the MySQL package. Embarassed

Fearfully I re-installed Exim, courier and lighttpd. Exim and courier were one line commands (apt-get install exim). To reinstall lighttpd (an upgraded version from the one I had before) I think I needed to copy and paste five lines or so from the Lighttpd Install Guide. Smile

Everything worked exactly as it did before, no harm done. phew

Perhaps you've guessed what's coming next. Buried inside the obscure error messages above is Process exim4. It was exim that was causing the problems, and by sheer serendipity uninstalling and reinstalling it fixed the problem.

So the error messages from Debian were very obscure, but uninstalling and reinstalling large packages was impressively and magnaminously straightforward.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-06-28 13:23:47 | |

Categories: ,


Movable Python Preview

emoticon:black_hat I'm working on an updated version of Movable Python. There are a few changes to the way it runs code, but most of the improvements are adding functionality to the GUI. There is probably two or three weeks work ahead to get it all working. Below is a mock up of the new GUI. I will do some usability tests before releasing, but if anyone has any suggestions for improvements on the proposed layouts, they would be appreciated :

The New Movable Python

In case you can't tell, instead of using pack I'm using the grid geometry manager to layout the GUI. It makes life so much easier, and is very simple to use. Also on the list for the future is integrating Tile so that Movable Python can acquire a native look and feel.

Oh and I added a picture of Mutex the Build Cat to a previous post. Smile

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-06-25 19:02:54 | |

Categories: , ,


Avocados

emoticon:avocado This isn't the sort of post I normally make here, but you programmers are a wise and eclectic bunch; so I thought I might get a useful answer.

How do you tell when an avocado is ripe or nearly ripe ? Whenever we buy them it seems like a complete lottery as to which ripen quickly (and are gorgeous) and which remain forever hard. Ones that seem virtually identical in the shop, in terms of appearance and hardness, will turn out to be radically different when we come to eat them. Any of you got any hints or clues ?

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-06-25 16:04:40 | |

Categories:


Python & the Static Typing Debate

emoticon:cross At work I spent some time with a colleague integrating PyLint into our build process. For the packages we completed work on (fixing the warnings), it now breaks the build to check in code with unused imports, unused variables or using undefined names.

It occurs to me that one [1] of the arguments against dynamic languages like Python, is that the compile phase will catch certain kinds of errors.

By integrating PyLint into our development process, we are now catching a lot of these errors (like typos for example, using undefined names because you've misspelt the variable [2]). You can also configure PyLint to require all instance attributes to be initialised in the __init__ method. This is reminiscent of requiring you to declare your variables, but would catch you misspelling attribute names.

The other big sort of error that compilers catch (by enforcing rigid typing), is type errors. If your function returns a float and you treat it as a string, your program won't compile. In Python this will compile, but blow up at runtime.

It occurs to me that over the years, particularly recent years, there has been a lot of research into Python and typing. This has resulted in a dramatic absence of any type analysis tools useful to the programmer. Smile

Relevant projects include :

In fact I believe that part of Brett Cannon's thesis involved running an analysis tool over the whole of the standard library. It was able to pick up blatant type errors in code. (Of which there were very few.)

It looks like type inferencers aren't able to provide a speed improvement without modifying the way Python works. For example, you can change the type of any variable from outside the module it is used in. (module.foo = bar.) However, analysis tools that can find obvious errors, and provide warnings about probable errors, may be a useful check in production environments. They would also remove a possible barrier for entry amongst those who like type safety as a precaution.

It seems a shame that so much work has been done by clever people, yet has resulted in little that is actually usable by programmers [3].

[1]Another argument is that static typing makes it possible to write faster code.
[2]Actually our test code would almost certainly pick this up, the other checks are more useful for us. We will also be enabling more checks as we go.
[3]PyPy is an exception, hopefully type annotation of RPython will be very useful soon; but it possibly has an even broader application.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-06-25 16:00:11 | |

Categories: ,


Movpy, Tkinter & Icons

emoticon:movpy2 I've finally got around to working on Movable Python.

I'm making fairly drastic changes, both in appearance and in extending its capabilities.

Some of the changes involve adding icons to buttons to improve the appearance of the GUI, which will hopefully do more whilst looking less cluttered.

I found several good sets of free program icons. There are some good image collections on the Maxpower List. In the end I have used a set of application and website images provided by glyFX.

The license for use states the icons must be compiled into the program rather than redistributed. I would probably have done this anyway.

I've stuck with Tkinter for the GUI, because it minimizes the dependencies for a minimal distribution (although wxPython is included with Movable Python). Having recently worked with both wxPython and Windows Forms, I actually find Tkinter less intuitive to use these days. It's not exactly difficult though.

I've found the script img2pytk.py, from 3D Artist Tkinter Notes, very helpful in converting GIF images into base64 strings for including as program data. The output isn't ideal, but it doesn't require more than a little editing.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-06-25 15:50:20 | |

Categories: ,


Python Just Keeps Racing

emoticon:python It seems like every few days there is fresh and interesting news. Python is alive and healthy, despite the competition from younger languages like ruby.

  • PyPy 0.9.0 Released

    Another hot from the oven release from the PyPy crew. It is several times faster than the last release, and now stable enough to run the full Python test suite to the end.

    Using a tool called ext-compiler it is possible to write extension modules compatible with both CPython and PyPy.

    It looks like they are finally getting to the point where they can implement the JIT compiler for PyPy. This is where they expect to make the real speed benefits.

  • Restricted Execution for Python

    In the dim and distant past a module called rexec (now deprecated), provided a restricted execution model for Python. This meant that you could sandbox a Python interpreter and run untrusted code.

    Unfortunately there proved to be just too many holes to patch and the rexec withered. This means that Python is unsuitable for using as a scripting language in browsers, and many other situations where restricted execution is needed.

    Brett Cannon has just announced that as part of his dissertation for getting Python into Firefox (to replace Javascript), he has created a design doc for restricted execution.

    If it works out he is looking to get this into Python 2.6. There are many places where this sandboxing is needed if Python is to be used, so it is a very welcome development.

  • Switch & Case

    A conspicuously missing feature from the Python language is the switch and case construction, common in many other languages.

    We seem to have got on fine without it so far, using dictionaries of functions or if/elif/else... chains to do the same job.

    case and switch can be easier to read, as well as offering possible compiler optimisations, where all the case statements only need to be evaluated once. There are several modules in the Python library that could benefit from this.

    However the issues in implementing this (and indeed deciding a syntax) turn out to be more mind-bending than you could imagine. Smile

    If you can follow the debate in this thread on Python-Dev, you're a better man than I. It looks like Guido is coming down in favour of something like this.

    • So we may gain a new static keyword (which only does anything inside a function body and is evaluated at function definition time).
    • All case expressions in a switch have an implied static.
    • A switch is implemented using a dict which is precomputed at the same time its static expressions are precomputed. The switch expression must be hashable.

    This sounds a bit convoluted to me, and adds new potential confusions about evaluation times. However it does add a flexible construct to the language, and allows for optimisations which are always welcomed. Smile

    You can get more information from Fredrik Lundh's micro-PEPs on Switch & Static Expressions.

From the sig on the PyPy announcement Smile :

I can't see a conspicuous evolutionary advantage in being good at higher mathematics.

—James Riden, asr

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-06-25 15:19:28 | |

Categories:


Hosted by Webfaction

Counter...