Python Programming, news on the Voidspace Python Projects and all things techie.

rest2web in Print

emoticon:bugs My English speaking readers may or may not know that there is a relatively new Python book, called Programmation Python written in French [1].

Of course this is good news in itself. Especially interesting to me is the reference on page 523 of the Contents Page (pdf) to rest2web. Intriguing, looks like rest2web has made it into print ! Laughing

Now, where can I find a Frenchman with a copy of the book ? Hmmm...

[1]The book also has it's own website

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-02-15 14:31:32 | |

Categories: , ,

Movable Python Enhancements, Coming Soon...

emoticon:cyberpunk Having rolled out Movable Python 1.0.1, I will soon start work on version 1.1.0. Yes, already. Surprised

Many of the proposed features are thanks to suggestions from Torsten Will.

The main change will be the release of a demo version of Movable Python. This will probably be in the form of a time limited distribution for Python 2.3, with an annoying nag on startup.

The feature enhancements will include some of the following :

  • An option to run -m commands from the GUI.
  • Tooltips for the GUI.
  • Make the options available from a menu.
  • Add a configurable tool menu.
  • Fix the from __future__ issue.
  • Addition of an XML library to the distribution.

Plus various other possibilities of course. There are several items on the TODO List that may be incorporated.

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-02-15 09:42:21 | |

Categories: ,

Movable Python 1.0.1

emoticon:movpy2 I'm pleased to be able to announce the release of Movable Python 1.0.1. This includes the release of Movable Python for Python 2.2.

To obtain it, visit the Movable Python Shop.

Existing users of Movable Python can go to the groups area and simply download an updated copy.

What's New ?

Added support for the -m command line option. This works for modules contained within the, as well as other modules on sys.path.

Movable Python now ignores (with warnings) the unsupported Python command line options.

Error messages are now printed to sys.stderr.

Can run '.pyc' and '.pyo' files. now run when launching an interactive interpreter.

Renamed the function to go interactive to interactive_mode.

New variable available in interactive, this is True if launching an interactive interpreter. This allows you to customize the environment differently for interactive sessions.

Add the lib directory (etc) to sys.path before entering interactive mode. (BUGFIX)

pywin32 extensions now import properly (even on machines that don't have them installed). (BUGFIX) Embarassed

Added the extra files for Firedrop2 Macros.

Changes for Python 2.2 compatibility. Distribution for Python 2.2 built.

What is Movable Python ?

Movable Python is a distribution of Python for Windows that doesn't need to be installed. It easily fits onto a USB memory stick. Python on a stick.

It is integrated with SPE, the Python IDE, to make Movable Python a portable Build, Test, and Run environment. It has a nice GUI to launch programs and control its behaviour.

Movable Python is useful in the following situations:

  • Machines where you can't install programs.
  • Where you need a portable 'Build, Test, and Run' Python environment.
  • Having several versions of Python on the same machine for forward/backward compatibility testing.
  • Easily deploying Python scripts without having to install Python.
  • Try before you buy - test Python without having to install it, including new versions .
  • 'Python Runtime Environment'. '.py' files can be associated with movpy.

For more information, see An Introduction to Movable Python.

Known Issues

  • There are problems with the pywin32 extensions and Python 2.2. They are included but may not all function correctly.
  • -m doesn't work with modules in under Python 2.2. This may or may not be resolvable.
  • Movable Python doesn't yet handle from __future__ import ... statements in scripts that are run. This will be fixed.

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-02-15 09:30:29 | |

Categories: ,

Relative Imports

emoticon:fish PEP 328 proposes a change to imports, specifically it addresses the problem of relative imports. This PEP addresses the problem of relative imports.

Currently modules are able to import from other modules within the same package as themselves using the form :

import foo

This becomes a problem if foo is a module name that clashes with a module name in the standard library (or indeed elsewhere on sys.path). Does the import refer to the neighbouring module, or the standard library module ? [1]

Of course, you can always fix this ambiguity by making import statements absolute :


PEP 328 resolves this uncertainty by making all (current) import statements absolute, and introducing a new syntax for relative imports. This PEP has been accepted for inclusion in Python 2.5, the deed is done [2].

From then on import foo will always be interpreted as referring to some module on sys.path.

You will be able to specify a module from within a package (a relative import) by using the new syntax :

import .foo

The upshot is that any package using relative imports will break in Python 2.5. Confused

Some of my code uses this, and some of the code I've inherited also uses this. So change all the code to the new relative syntax ? Nope, this is a syntax error in any version of Python pre-2.5. That means you can't even conditionally use the new syntax. sigh

I guess it's not more than a few hours work to scan all my codebase for relative imports and change them to absolute ones, it's just a few hours I'd rather have been doing something else.

I wonder if that does leave me with a problem though. Some packages I do put into sys.path. If pythonutils is installed using distutils it creates a .pth file that adds the pythonutils package directory to sys.path. This means that modules can use relative imports (because they're not really relative at all) [3]. But it also makes the sub-modules available through the pythonutils namespace. In distributed code you could just include the pythonutils directory with your application and do import pythonutils.configobj. You won't be able to do that now without adding the pythonutils directory to sys.path, because the configobj module has a bare import validate in it. (Which previously succeeded as a relative import and an absolute import.) Maybe I'll have to do some magic in to make names available.

I doubt this will break much code, but it might be the cause of a frustrating bug-hunt for someone out there.

[1]The answer depends on which path comes first in sys.path.
[2]The PEP implies that at least some of this functionality is already available in Python 2.4, by specifying from __future__ import absolute_import. I haven't tried this.
[3]If the pythonutils directory is on sys.path then one pythonutils module that uses another one is doing it via absolute imports. If the directory isn't on sys.path then the same import statement does the same thing as a relative one.

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-02-14 16:03:38 | |

Categories: ,

Firedrop2 is On the Move

emoticon:firedrop2 Firedrop2 (the Python blog client) has a new developer, and he's contributing so many changes my head is spinning.

Stewart Midwinter has started using Firedrop2 and made many fixes already.

Firedrop2 is a great blog client, with the usual lineup of features. You completely control the blog from your own computer (the client), and it generates static HTML pages which it can even FTP to your site. This means that your data is always under your control.

Unfortunately, up until now, it has only really been suitable for Python programmers. Errors filter through to you as Python tracebacks in a dialog, even basic errors like trying to create a new entry without having a blog open (Firedrop2 allows you to maintain several blogs, as well as other document collections). Some entry metadata is also stored within entries as text, using Python syntax (e.g. the category list as a list of strings, quotes'n'all).

These are all reasonably easily resolved but I haven't had the time. Stewart has started to work his way through the list, and it's nice to see this project start to move. Straight after I've pushed out the new Movable Python release, I'll start to work through the changes he's made and do a new release. Smile

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-02-13 21:14:45 | |

Categories: ,

Simulating '-m' Part II

emoticon:firefox This is a more advanced way of simulating the -m Python command line option.

This version supports modules in zipfiles, using zipimport. It can also load .pyc files.

It returns a code object, which you can then pass to the exec statement. Smile

import os, sys
import zipimport
import marshal

def modulefinder(mod):
    Given a module name, return a code object. This is used to simulate the
    ``-m`` Python command line option. You can then use ``exec`` on the
    resulting code object.

    Works for modules with an associated file of the following types :

        ``('.pyc', '.pyo', '.py', '.pyw')``

    If it can't find the associated file on the path, it raises a

    This function could raise a ``SyntaxError`` if the ``compile`` fails.

    It could raise a ``ValueError`` if un-marshaling a '.pyc' file fails.

    It will find files in a zip file (using zipimporter), but *can't* find
    modules in packages in zip files.

    If a package name is specified, with no sub-package, then it will not be
    found. (We could change this to use the ```` file of the

    sources = ('.py', '.pyc', '.pyo', '.pyw')
    # Is it a package file ?
    mods = mod.split('.')
    mod1, mods = mods[0], mods[1:]
    for entry in sys.path:
        if entry.endswith('.zip') and not mods:
            # Need to use zipimporter
                z = zipimport.zipimporter(entry)
            except zipimport.ZipImportError:
                # not a valid zipfile
                # we'll try this path as a directory
                    return z.get_code(mod)
                except zipimport.ZipImportError:
        # try it as a directory
        if mods and os.path.isdir(os.path.join(entry, mod1)):
            # a package
        elif not mods:
            for ending in sources:
                if os.path.isfile(os.path.join(entry, mod1 + ending)):
                    path = os.path.join(entry, mod1 + ending)
                    code = open(path, 'rb').read()
                    if ending in ('.pyc', '.pyo'):
                        return marshal.loads(code[8:])
                        code = code.replace('\r\n', '\n') + '\n'
                        return compile(code, path, 'exec')
        raise ValueError('No file associated with module "%s".' % mod)
    last, mods = mods[-1], mods[:-1]
    the_path = os.path.join(entry, mod1, *mods)
    for ending  in sources:
        path = os.path.join(the_path, last + ending)
        if os.path.isfile(path):
            code = open(path, 'rb').read()
            if ending in ('.pyc', '.pyo'):
                return marshal.loads(code[8:])
                code = code.replace('\r\n', '\n') + '\n'
                return compile(code, path, 'exec')
    raise ValueError('No file associated with module "%s".' % mod)

if __name__ == '__main__':
    # should print the code object for email.test.test_email
    print modulefinder('email.test.test_email')
    # 'Should fail to find this fictitious file.'
        print modulefinder('')
    except ValueError:
        print '****Failed to find "" ****'
    # should print the code object for base64
    print modulefinder('base64')
    # should *fail* to find email - because it is a package
        print modulefinder('email')
    except ValueError:
        print '****Failed to find "email" ****'

This code is unlikely to be directly useful to anyone, but it is potentially interesting as :

  • An example of extracting code objects from bytecode files
  • An example of searching sys.path for a module
  • An example of how to use the zipimport module
  • An example of using compile - note that you have to get rid of occurences of \r\n and terminate with a \n

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-02-12 17:27:19 | |

Categories: ,

Bytecode to Code Object

emoticon:dove Question: How do you turn a .pyc (bytecode) file back into the code object that it contains ?

Answer: Google a lot, and eventually discover a blog entry by Bob Ippolito on Python bytecode files. This tells me that the marshalled code object is stored from the ninth byte onwards... Smile

import marshall
data = open(filename, 'rb').read()
code_data = data[8:]
code_object = marshall.loads(code_data)
"You can now pass the code object to exec."

Looks like Movable Python can now run .pyc files. I actually needed this to support the -m option (running a module instead of importing it).

I have discovered an issue with Movable Python. It's not one that's bitten me yet, but may byte (sorry) someone. The documentation on the compile function mentions the optional flags argument to compile. This allows you to specify which __future__ statements your code is compiled with. The default is to use the same ones as the code calling compile.

I've ignored this so far, which means that future statements are probably broken. Sad

I either need to scan code for future statements prior to compiling, or provide an option to support them in another way. This will have to wait until the release after the one I'm about to do.

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-02-12 17:05:47 | |

Categories: , ,

Simulating '-m'

emoticon:carrot I've been implementing a simple simulation of the -m command line option for Movable Python.

It finds a module on the path, and runs it as a script. Here is a basic implementation. It only works if the module is a .py file. It does work with packages, so long as they don't do anything magic :

def modulefinder(mod):
    Given a module name, return the path to the file.

    Only works for modules with an associated '.py' file.

    Will raise a ``ValueError`` if it can't find the '.py' file.

    # Is it a package file ?
    mods = mod.split('.')
    mod1, mods = mods[0], mods[1:]
    if mods:
        package = True
        package = False
    for entry in sys.path:
        if package and os.path.isdir(os.path.join(entry, mod1)):
        elif not package and os.path.isfile(os.path.join(entry, mod1 + '.py')):
        raise ValueError('No file associated with module "%s".' % mod)
    if not package:
        return os.path.join(entry, mod1 + '.py')
    last, mods = mods[-1], mods[:-1]
    the_path = os.path.join(entry, mod1, *mods)
    the_path = os.path.join(the_path, last + '.py')
    if not os.path.isfile(the_path):
        raise ValueError('No file associated with module "%s".' % mod)
    return the_path

Now if a 'package' is specified, should it return the file of that package ? Currently it raises an error. Python 2.4.2 also raises an error, but I believe they are about to change the way -m works for Python 2.5.

This implementation doesn't work with zipimport, which may limit it's usefulness for Movable Python. Anyway, the relevant PEP is PEP-0338.

Like this post? Digg it or it.

Posted by Fuzzyman on 2006-02-12 10:47:45 | |

Categories: , ,

Hosted by Webfaction