Python Programming, news on the Voidspace Python Projects and all things techie.
Linux is Great, I Think
From 6pm Monday until around 9pm yesterday (Tuesday) my server was mainly down. This meant no email and no website.
I could restart the server, but it would only last a few minutes before wedging itself thoroughly again.
When I logged in via SSH, I got obscure messages streaming down the screen :
Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: Oops: 0000 [#1] Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: SMP Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: CPU: 0 Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: EIP is at _mmx_memcpy+0x92/0x125 Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: eax: 00000000 ebx: 03ffe240 ecx: c25ecc94 edx: c6a9e000 Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: esi: c6b05fc4 edi: c2c681cc ebp: ffff0c7c esp: c252b68c Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: ds: 007b es: 007b ss: 0069 Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: Process exim4 (pid: 7027, threadinfo=c252a000 task=cb8a9030) Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: Stack: <0>c2c0058c c6a9e384 ffff0c7c 00000138 c019984f c2c0058c c6a9e384 ffff0c7c Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: ffff0c7c c298cd54 c252b710 c2bf1180 c25ecc94 00000010 00000070 00000010 Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: 0000f58c 00000208 c2bf1000 ffffff38 0000000d c298cd54 00000000 c252b710 ... Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: [<c015a426>] __bread+0x6a/0x7e Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: [<c01a21dd>] reiserfs_prepare_for_journal+0x63/0x6a Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: [<c0193920>] clear_all_dirty_bits+0x9/0xd Message from syslogd@voidspace at Mon Jun 26 19:10:57 2006 ... voidspace kernel: [<c019eb68>] reiserfs_delete_item+0xf4/0x32a
I missed a vital clue buried herewithin and tried all sorts of things. Updates didn't fix it, top showed no rogue processes, chkrootkit came up clean, memory, disk and CPU usage were all normal. I finally spoke to a knowledgeable friend who suggested it wass likely to be a hardware problem, so I tried memtest (which didn't reveal anything) and raised a support ticket.
In the meantime I decided to clear up some stray packages which I'd installed and wasn't really using, especially MySQL which was originally installed for someone else. I spent a while trying to persuade apt-get to list all installed packages, before discovering that the required magic incantation is dpkg -l.
I could see various packages related to MySQL installed, including ones like mysql-common. This sounded like a good place to start, and without properly reading the messages I started the remove. To my horror I watched apt-get faithfully obey my instructions and proceed to uninstall Exim (my mail transfer agent), lighttpd (my webserver), courier (my POP mail server) and more. All of these depended on the MySQL package.
Fearfully I re-installed Exim, courier and lighttpd. Exim and courier were one line commands (apt-get install exim). To reinstall lighttpd (an upgraded version from the one I had before) I think I needed to copy and paste five lines or so from the Lighttpd Install Guide.
Everything worked exactly as it did before, no harm done. phew
Perhaps you've guessed what's coming next. Buried inside the obscure error messages above is Process exim4. It was exim that was causing the problems, and by sheer serendipity uninstalling and reinstalling it fixed the problem.
So the error messages from Debian were very obscure, but uninstalling and reinstalling large packages was impressively and magnaminously straightforward.
Movable Python Preview
I'm working on an updated version of Movable Python. There are a few changes to the way it runs code, but most of the improvements are adding functionality to the GUI. There is probably two or three weeks work ahead to get it all working. Below is a mock up of the new GUI. I will do some usability tests before releasing, but if anyone has any suggestions for improvements on the proposed layouts, they would be appreciated :
In case you can't tell, instead of using pack I'm using the grid geometry manager to layout the GUI. It makes life so much easier, and is very simple to use. Also on the list for the future is integrating Tile so that Movable Python can acquire a native look and feel.
Oh and I added a picture of Mutex the Build Cat to a previous post.
This isn't the sort of post I normally make here, but you programmers are a wise and eclectic bunch; so I thought I might get a useful answer.
How do you tell when an avocado is ripe or nearly ripe ? Whenever we buy them it seems like a complete lottery as to which ripen quickly (and are gorgeous) and which remain forever hard. Ones that seem virtually identical in the shop, in terms of appearance and hardness, will turn out to be radically different when we come to eat them. Any of you got any hints or clues ?
Python & the Static Typing Debate
At work I spent some time with a colleague integrating PyLint into our build process. For the packages we completed work on (fixing the warnings), it now breaks the build to check in code with unused imports, unused variables or using undefined names.
It occurs to me that one  of the arguments against dynamic languages like Python, is that the compile phase will catch certain kinds of errors.
By integrating PyLint into our development process, we are now catching a lot of these errors (like typos for example, using undefined names because you've misspelt the variable ). You can also configure PyLint to require all instance attributes to be initialised in the __init__ method. This is reminiscent of requiring you to declare your variables, but would catch you misspelling attribute names.
The other big sort of error that compilers catch (by enforcing rigid typing), is type errors. If your function returns a float and you treat it as a string, your program won't compile. In Python this will compile, but blow up at runtime.
It occurs to me that over the years, particularly recent years, there has been a lot of research into Python and typing. This has resulted in a dramatic absence of any type analysis tools useful to the programmer.
Relevant projects include :
A static type inferencer for Python. Never released.
A thesis by Brett Cannon which concluded that static type inferencing for Python wouldn't offer significant speed advantages.
Type annotation is a crucial part of compiling RPython programs, in PyPy.
A slightly dated paper by John Aycock, explaining his experiments with a Python to Perl translator. (Where Perl needs type specifiers on variables.)
In fact I believe that part of Brett Cannon's thesis involved running an analysis tool over the whole of the standard library. It was able to pick up blatant type errors in code. (Of which there were very few.)
It looks like type inferencers aren't able to provide a speed improvement without modifying the way Python works. For example, you can change the type of any variable from outside the module it is used in. (module.foo = bar.) However, analysis tools that can find obvious errors, and provide warnings about probable errors, may be a useful check in production environments. They would also remove a possible barrier for entry amongst those who like type safety as a precaution.
It seems a shame that so much work has been done by clever people, yet has resulted in little that is actually usable by programmers .
|||Another argument is that static typing makes it possible to write faster code.|
|||Actually our test code would almost certainly pick this up, the other checks are more useful for us. We will also be enabling more checks as we go.|
|||PyPy is an exception, hopefully type annotation of RPython will be very useful soon; but it possibly has an even broader application.|
Movpy, Tkinter & Icons
I've finally got around to working on Movable Python.
I'm making fairly drastic changes, both in appearance and in extending its capabilities.
Some of the changes involve adding icons to buttons to improve the appearance of the GUI, which will hopefully do more whilst looking less cluttered.
The license for use states the icons must be compiled into the program rather than redistributed. I would probably have done this anyway.
I've stuck with Tkinter for the GUI, because it minimizes the dependencies for a minimal distribution (although wxPython is included with Movable Python). Having recently worked with both wxPython and Windows Forms, I actually find Tkinter less intuitive to use these days. It's not exactly difficult though.
I've found the script img2pytk.py, from 3D Artist Tkinter Notes, very helpful in converting GIF images into base64 strings for including as program data. The output isn't ideal, but it doesn't require more than a little editing.
Python Just Keeps Racing
It seems like every few days there is fresh and interesting news. Python is alive and healthy, despite the competition from younger languages like ruby.
Another hot from the oven release from the PyPy crew. It is several times faster than the last release, and now stable enough to run the full Python test suite to the end.
Using a tool called ext-compiler it is possible to write extension modules compatible with both CPython and PyPy.
It looks like they are finally getting to the point where they can implement the JIT compiler for PyPy. This is where they expect to make the real speed benefits.
In the dim and distant past a module called rexec (now deprecated), provided a restricted execution model for Python. This meant that you could sandbox a Python interpreter and run untrusted code.
Unfortunately there proved to be just too many holes to patch and the rexec withered. This means that Python is unsuitable for using as a scripting language in browsers, and many other situations where restricted execution is needed.
If it works out he is looking to get this into Python 2.6. There are many places where this sandboxing is needed if Python is to be used, so it is a very welcome development.
A conspicuously missing feature from the Python language is the switch and case construction, common in many other languages.
We seem to have got on fine without it so far, using dictionaries of functions or if/elif/else... chains to do the same job.
case and switch can be easier to read, as well as offering possible compiler optimisations, where all the case statements only need to be evaluated once. There are several modules in the Python library that could benefit from this.
However the issues in implementing this (and indeed deciding a syntax) turn out to be more mind-bending than you could imagine.
- So we may gain a new static keyword (which only does anything inside a function body and is evaluated at function definition time).
- All case expressions in a switch have an implied static.
- A switch is implemented using a dict which is precomputed at the same time its static expressions are precomputed. The switch expression must be hashable.
This sounds a bit convoluted to me, and adds new potential confusions about evaluation times. However it does add a flexible construct to the language, and allows for optimisations which are always welcomed.
You can get more information from Fredrik Lundh's micro-PEPs on Switch & Static Expressions.
From the sig on the PyPy announcement :
I can't see a conspicuous evolutionary advantage in being good at higher mathematics.
—James Riden, asr
This work is licensed under a Creative Commons Attribution-Share Alike 2.0 License.