Voidspace

A Little Bit of Python: Episodes 5 and 6

Two more episodes of A Little Bit of Python have been posted. A Little Bit of Python is an occasional podcast on Python related topics with myself, Brett Cannon, Jesse Noller, Steve Holden and Andrew Kuchling.

Episode 5 is a round table discussion of some of the new things coming in Python 2.7. This includes a discussion of the changes coming to the logging module and distutils (with rambling side discussions at every point of course). Episode 6 is the first of a series of interviews I recorded at PyCon. This interview is with Van Lindberg, who lists amongst his achievements being the PyCon chair this year [1], being the PSF lawyer and a PSF member, and also the author of Intellectual Property and Open Source. In his spare time he works as an intellectual property lawyer. In the interview, recorded after PyCon 2010 had finished, we discuss how PyCon went, what we can look forward to in PyCon 2011 and a bit about his book and his work.

• Episode 5: A Little Bit of Python (mp3)
• Episode 5: A Little Bit of Python (m4a)
• Episode 6: A Little Bit of Python (mp3)
• Episode 6: A Little Bit of Python (m4a)

General links for the podcast feeds and a webpage with an embedded flash player:

• A Little Bit of Python mp3 rss feed
• A Little Bit of Python m4a rss feed
• Podcast homepage (currently redirecting to a temporary home)

If you have feedback, insults or suggestions for new topics you can email us on: all@bitofpython.com. We don't yet have the podcast listed on iTunes; we'll set that up once our permanent online home goes live.

We do have a twitter account, so for news on new episodes follow @bitofpython. A Little Bit of Python is also syndicated on Hacker Public Radio (although they're only up to episode two so far).

Encoding json on Silverlight with System.Json

At the backend of last I year I wrote a blog entry on decoding json on Silverlight. Well, the time has finally come and we're now encoding json to post back to our Django application. The code to encoode json is just as simple as the code for decoding. It makes extensive use of clr.Convert(...) as encoding json relies heavily on implicit conversions:

import clr
from System.Collections.Generic import Dictionary
from System.Json import JsonValue, JsonArray, JsonObject

NoneType = type(None)

def encode_string(data):
    return encode_object(data).ToString()

def encode_object(value):
    if isinstance(value, (int, long, bool, float, str, NoneType)):
        # JsonValue has implicit conversion operators for all
        # these types
        return clr.Convert(value, JsonValue)
    elif isinstance(value, (list, tuple, set)):
        return JsonArray([encode_object(obj) for obj in value])
    elif isinstance(value, dict):
        values = Dictionary[str, JsonValue]()
        for key, val in value.items():
            values[key] = encode_object(val)
        return JsonObject(values)

    # raise an error if we encounter an unhandled type
    raise TypeError(value)

encode_string takes any data-structure of Python primitive / container types and serializes it into JSON. This json seems to be consumed fine by simplejson on the other end. As well as being simple this code is blazingly fast. The actual encoding operation is done in the .ToString() call, which all happens in C#.

When you send the json over the wire then it will be encoded as UTF-8 for the simple reason that this is the only encoding that Silverlight ships with.

This is relevant because the reason we are shipping huge data structures over the wire is in order to export Excel files from our data-grid views. I initially tried to use xlwt inside Silverlight. Unfortunately it relies on either latin-1 or utf-16 encodings and so my efforts were doomed to failure. Fortunately it works grand on the server side, but the encoding restriction does place limits on the Python libraries that you can use in the browser (unless you are prepared to write your own encoding functions of course).

The all new and improved unittest: unittest2

At PyCon I gave a talk on some of the new features that are coming to unittest.

unittest is the Python standard library testing framework. In recent years unittest has languished whilst other Python testing frameworks like nose and py.test have innovated. Some of the best innovations have made their way into unittest which has had quite a renovation. In Python 2.7 & 3.2 a whole bunch of improvements to unittest will arrive.

The video of my talk (30 minutes) is up online:

• New *and* Improved: Coming changes to unittest

I've turned the slides of the talk into an article:

• Coming changes to unittest in Python 2.7 & 3.2

The talk / article introduces unittest2, a back-port of the features in unittest to work with Python 2.4, 2.5 & 2.6. unittest2 is developed with a Mercurial repository on hg.python.org/unittest2 [1]. unittest2 is already in use for the development of distutils2 and there is an open ticket to make use of it in Django.

• unittest2 on PyPI
• unittest2 mercurial repo

The new features include:

• automatic test discovery and new command line features
• addCleanups - better resource management
• many new assert methods including better defaults for comparing lists, sets, dicts unicode strings etc and the ability to specify new default methods for comparing specific types
• assertRaises as context manager, with access to the exception object afterwards
• test skipping and expected failures
• load_tests protocol for controlling the loading of tests from modules or packages
• startTestRun and stopTestRun methods on TestResult
• various other API improvements and fixes

You can install unittest2 with:

pip install unittest2

To use it in your modules change import unittest to import unittest2. Alternatively you can conditionally make use of it where it is available:

try:
    import unittest2 as unittest
except ImportError:
    import unittest

Please use unittest2 and report bugs, or make feature requests, on the unittest2 issue tracker or the Python issue tracker.

New Release: ConfigObj 4.7.2

A new version of ConfigObj has just been released: 4.7.2

This is a bugfix release with several important bugfixes. It is recommended that all users of ConfigObj 4.7 update.

• Direct Download Link
• PyPI Page
• ConfigObj Home Page & Documentation

The new version can be installed with:

pip install -U configobj

Changelog

• BUGFIX: Restore Python 2.3 compatibility
• BUGFIX: Members that were lists were being returned as copies due to interpolation introduced in 4.7. Lists are now only copies if interpolation changes a list member. (See below.)
• BUGFIX: pop now does interpolation in list values as well.
• BUGFIX: where interpolation matches a section name rather than a value it is ignored instead of raising an exception on fetching the item.
• BUGFIX: values that use interpolation to reference members that don't exist can now be repr'd.
• BUGFIX: Fix to avoid writing '\r\r\n' on Windows when write is given a file opened in text mode ('w').

See below for important details on the change to using list values with string interpolation.

What is ConfigObj?

ConfigObj is a simple but powerful config file reader and writer: an ini file round tripper. Its main feature is that it is very easy to use, with a straightforward programmer's interface and a simple syntax for config files. ConfigObj has a host of powerful features:

• Nested sections (subsections), to any level

• List values

• Multiple line values

• Full Unicode support

• String interpolation (substitution)

• Integrated with a powerful validation system

  • including automatic type checking/conversion
  • and allowing default values
  • repeated sections

• All comments in the file are preserved

• The order of keys/sections is preserved

• Powerful unrepr mode for storing/retrieving Python data-types

String Interpolation and List Values

For general information on string interpolation in ConfigObj see: String Interpolation in ConfigObj.

Since version 4.7 string interpolation is done on string members of list values. If interpolation changes any members of the list then what you get back is a copy of the list rather than the original list.

This makes fetching list values slightly slower when interpolation is on, it also means that if you mutate the list changes won't be reflected in the original list:

>>>  c  =  ConfigObj()
>>>  c['foo']  =  'boo'
>>>  c['bar']  =  ['%(foo)s']
>>>  c['bar']
['boo']
>>>  c['bar'].append('fish')
>>>  c['bar']
['boo']

Instead of mutating the list you must create a new list and reassign it.

dict.__getitem__(configobj, key)

An Awesome PyCon

As always PyCon was awesome, I had a great time catching up with old friends and making some new friends. The conference wasn't bad either...

This year was particularly awesome. After a recession fuelled downturn last year, this year numbers were back up to a recording breaking level. We had around 1025 attendees, and importantly a higher than ever proportion of women. Although 11% is hardly stellar it is much higher than previous years and better than many large open source conferences. This is still an underrepresentation of the number of women in IT, but still a great improvement. Much kudos to Gloria Willasden and Steve Holden who put a lot of effort into promoting PyCon to women developers and creating the Financial Assistance Grant for Women.

This year was the year of alternative implementations, with a lot of focus on Unladen Swallow, PyPy, Jython, IronPython and even Pynie (a fledgling implementation of Python on the Parrot virtual machine). Nice to see that PyPy have just posted some benchmarks showing that they are largely faster than Unladen Swallow and CPython running Twisted. Performance is a hot topic for dynamic languages in general, and there is healthy competition amongst the implementations for speed. One of the things to come out of the Language Summit is a standard repository of benchmarks for implementations to share.

I was responsible for organising the Python Language Summit immediately preceding the conference. I think it went well, but as far as I'm concerned if it wasn't a complete disaster then it counts as a great success! At the summit we spent a lot of time discussing packaging. The upshot of this was a new way of handling the transition to an improved distutils - create a new module called distutils2 that is free to break backwards compatibility and then bring this into the standard library. Tarek has been working on improving distutils over the last year whilst maintaining backwards incompatibility. The problem is that so many tools depend on, extend and monkeypatch distutils in so many ways (including relying on buggy behaviour) that this has been virtually impossible. The new approach, a clean break, is good but means a sudden change of direction of Tarek. He has written this up, including the progress he and his team have already made: The Fate of distutils.

During the language summit Guido also approved 3 PEPs, all important in their own way:

• PEP 389: argparse - new command line parsing module

  argparse is now in the standard library.

• PEP 3147: PYC Repository Directories

  The way Python creates and stores its compiled bytecode files (.pyc) is changing.

• PEP 3146: Merging Unladen Swallow into CPython (3)

  There are still a lot of issues to resolve before the final merge happens, but in principle the merge is now approved. The team, who includes several outside contributors making google employees a minority, need to improve startup time, reduce memory use and also make it faster - but it looks like Python 3.3 will have a JIT!

As with last year the audio visual team did an outstanding job of recording the talks, many of which (but not all at the time of writing) are available to watch online at pycon.blip.tv and the Python Miro Community. Some of the ones I recommend are:

• Four powerful examples of composing Python tools

  Raymond Hettinger at his best!

• Understanding the Python GIL

  Easily the star talk of the conference, standing room only. An extremely well presented talk on the Python Global Interpreter Lock by David Beazley.

• Python in the Browser

  Jimmy Schementi shows off running Python code in the browser with Silverlight - including <script type="text/python">...

• State of IronPython Keynote

  Dino Veihland shows off the new IronPython integration in Visual Studio 2010, including very cool intellisense that does type inferencing on your code.

There are lots of great talks there, with more arriving, so have a browse through.

This year I had a new toy at PyCon, a portable audio recorder that I used to record a bunch of interviews for A Little Bit of Python. I've recorded some very cool interviews, but they'll take a bit of editing and will go live over the next few weeks:

• Antoine Pitrou - CPython developer and creator of the new GIL
• Paul Hildebrandt - Senior Programmer at Disney Animation Studios
• Frank Wierzbicki - Lead developer of Jython
• Christian Tismer - Creator of Stackless Python, one of the original PyPy developers plus developer of version 2 of Pysco the specializing JIT for Python
• Allison Randall - Architect of the Parrot VM and lead developer of Pynie
• David Beazley - Python trainer and author of The Python Essential Reference
• Jacob Kaplan-Moss & James Tauber - BDFLs of Django and Pinax
• Jimmy Schementi, Dino Veihland, Dave Fugate and Bill Chiles - The IronPython team
• Tarek Ziade - Developer currently fixing Python packaging
• Jean-Paul Calderone and Jonathan Lange - Twisted core developers
• Collin Winter and Jeffrey Yaskin - Lead Unladen Swallow developers
• Maciek Fijalkowski - A core PyPy developer
• Van Lindberg - PSF lawyer, PyCon chair and author of Intellectual Property and Open Source

As soon as any of these go up I'll be sure to let you know.

A Little Bit of Python Episode 4: A Pre-PyCon Special

A Little Bit of Python is an occasional podcast on Python related topics with myself, Brett Cannon, Jesse Noller, Steve Holden and Andrew Kuchling.

The website is in progress and apparently nearly ready, thanks to Jesse and various other people who we will thank as soon as it is done. In the meantime, episode 4 is out. PyCon 2010 is only ten days away and it is the highlight of the year for many of us in the Python community. This episode is a pre-PyCon special where we discuss some of the things that will be happening at the conference and how to get the best out of it.

• Episode 4: A Little Bit of Python (mp3)
• Episode 4: A Little Bit of Python (m4a)

General links for the podcast feeds and a webpage with an embedded flash player:

• A Little Bit of Python mp3 rss feed
• A Little Bit of Python m4a rss feed
• Podcast homepage (currently redirecting to a temporary home)

If you have feedback, insults or suggestions for new topics you can email us on: all@bitofpython.com. We don't yet have the podcast listed on iTunes; we'll set that up once our permanent online home goes live.

We do have a twitter account, so for news on new episodes follow @bitofpython. A Little Bit of Python is also syndicated on Hacker Public Radio (although they've only released episode one so far).

ConfigObj 4.7.1 (and how to test warnings)

I hate doing releases. I haven't managed to automate the whole process (I should probably work on that), although setup.py sdist upload certainly helps. Anyway, the short version of the story (and the real reason I hate releases) is that was a bug in ConfigObj 4.7.0. 4.7.1 is a brown paper bag release to fix the bug

• ConfigObj 4.7.1 (.zip)
• ConfigObj Homepage
• ConfigObj on PyPI

The bug was an error in the way I had setup the deprecation warning for the obsolete options dictionary in the ConfigObj constructor. The bug only affects you if you were still using the options dictionary to configure ConfigObj instances.

If you've never heard of ConfigObj now is an ideal time to try it out. ConfigObj is a simple but powerful config file reader and writer: an ini file round tripper. Its main feature is that it is very easy to use, with a straightforward programmer's interface and a simple syntax for config files. It has lots of other features though :

• Nested sections (subsections), to any level

• List values

• Multiple line values

• Full Unicode support

• String interpolation (substitution)

• Integrated with a powerful validation system

  • including automatic type checking/conversion
  • and allowing default values
  • repeated sections

• All comments in the file are preserved

• The order of keys/sections is preserved

• Powerful unrepr mode for storing/retrieving Python data-types

The reason the bug happened is because not only did I not even try the deprecation warning to make sure it worked, let alone add a test for it. It turns out that adding a test for warnings is easy using the catch_warnings context manager, new in Python 2.6.

from warnings import catch_warnings
with catch_warnings(record=True) as log:
    ConfigObj(options={})

# unpack the only member of log
warning, = log
self.assertEqual(warning.category, DeprecationWarning)

Discover 0.3.2 and the load_tests protocol

discover is a test discovery module for the standard library unittest test framework. Test discovery is built into unittest in Python 2.7 and 3.2. The discover module is a back-port of test discovery to work with Python 2.4 - 2.6 and Python 3.0 / 3.1 [1].

3.2 is a minor bugfix release. Test discovery also includes a new protocol called load_tests. In previous versions the standard tests would be passed in as a list instead of a TestSuite instance. This bug is now fixed both in untitest and in discover.

• discover module on PyPI
• unittest-ext google code project (SVN)

discover can be installed with pip or easy_install. After installing switch the current directory to the top level directory of your project and run:

python -m discover
python discover.py

This will discover all tests (with certain restrictions) from the current directory. The discover module has several options to control its behavior (full usage options are displayed with python -m discover -h). See the documentation on the PyPI homepage for details.

The load_tests protocol is interesting. It allows you to customize how tests are loaded from a module by defining a load_tests function. load_tests takes three arguments, the test loader, the standard set of tests for that module (allowing you to just add tests to the standard set if you want). The third argument is only used for load_tests functions in the __init__.py of test packages.

Here's an example of a test module with two test classes. The load_tests function returns a test suite that only uses one of the test classes:

import unittest


class FirstTest(unittest.TestCase):
    def testFoo(self):
        self.fail()

class SecondTest(unittest.TestCase):
    def testFoo(self):
        pass

def load_tests(loader, tests, _):
    return loader.loadTestsFromTestCase(SecondTest)

When the test module is loaded by discover, or unittest from Python 2.7 / 3.2, then load_tests will be called to create the test suite for the module.

[1]	discover.py is only about 300 lines of Python. Supporting 2.x and 3.x in a single code-base is easy with small modules but I wouldn't recommend it for larger projects.

Interview with Pardus Linux

I recently did an interview on Python with the Pardus Linux magazine. Pardus Linux is a distribution developed in Turkey (by the Turkish National Research Institute of Electronics and Cryptology) with the goal of being usable by "normal" people rather than just geeks.

Pardus are great supporters and users of Python. A while ago they chose Python as their standard language for custom package and configuration management tools.

• Interview with Pardus Linux (in Turkish)

The magazine is Turkish, so here's the original in English. It covers topics like why use Python?, choosing a GUI toolkit, the move from Python 2 to 3 and whether there will ever be a Python compiler. The Turkish version is a very nicely produced PDF with waaay too many pictures of me, and even one of my wife.

• Who is Michael Foord?

  I've been developing with Python since 2002. I've written many articles on Python and maintain several open source projects. In 2006 I started working with a firm called Resolver Systems developing a highly programmable spreadsheet application with the then young IronPython. I wrote the book IronPython in Action with a colleague and was made the first Microsoft MVP (Most Valued Professional) for dynamic languages.

  In 2009 I became a core Python developer, Python Software Foundation (PSF) member and part of the Python.org webmaster team. I've also been involved in helping to organise both PyCon in the US and PyCon UK (which in 2009 and 2010 is hosting EuroPython).

  I'm now a freelance developer currently working with a German firm called Comsulting. We're developing web applications with Django, deployed on Linux of course, but with the client side written in IronPython on Silverlight.

  I blog and write a great deal about Python and other technical topics.

  In the real world I live in Northampton, in the UK, with my wife Delia.

• Which department of Python development are you working?

  I mainly help maintain the standard library, and in particular maintain the testing framework unittest. I've also helped with making parts of the standard library compatible with IronPython.

• Why Python? What's the main idea of development of Python?

  Python is a widely used programming language with a large(ish) code base. Parts of the code, in particular parts of the standard library are quite old. There are generally good tests, which are getting better, but we still get a lot of feature requests and bug reports. There is always more than enough work, and for me it is a privilege to be able to contribute back to the open source project that has given so much to me.

  On a more self-interested note I am a big fan of the unittest style of testing. However, other testing libraries like nose and py.test have advanced the state of the art of testing in Python a great deal whilst unittest has stagnated. It has been great to get some of the tried and tested features from other testing libraries, like test discovery, into the standard library.

  In the last few years Python has evolved a great deal, in particular the release of Python 3 which deliberately broke backwards compatibility with the Python 2 series to fix a number of longstanding 'warts' with the language. Python 3 is a great release, and comes with the full Python standard library, but third party libraries and the community are only gradually migrating to Python 3.

  The current versions of Python are 2.6 and 3.1, with both Python 2.7 and 3.2 releases imminent.

  Even without Python 3 a number of powerful new language features have been added in recent releases. From decorators to generators (and in particular the expansion of generators to provide coroutine functionality in Python 2.5) to conditional expressions, the with statement, abstract base classes and the introduction of new libraries like multiprocessing.

  All of this means that the Python community and developers need a chance to adapt to the new features. This is the reason for the Python Language Moratorium (PEP 3003) that mandates no new language features for the next two years. For the developers this means a chance to focus on the standard library rather than adding new language features.

• Why did you choose Python?

  I kind of got into Python by accident. I had done some programming, including assembly language on the Amiga, up to my time at university but I didn't really use computers for a while after that. A bunch of friends and I were playing a "Play By Email" strategy game called Atlantis and wanted to write a program to automate turns.

  Our one constraint on programming language was that there be an implementation for the PocketPC, which was the only computer I had at home at the time. We had more or less decided on Squeak (which would have been a fine choice), but at the last minute someone suggested Python and we went with it.

  I really enjoyed programming, and in particular Python, and before long had given up the game but got seriously hooked on Python.

• Why should people choose Python?

  As it happened Python was a fortuitous choice for me. Python is easy to learn for people new to the language, but is a fully fledged programming language suitable for large application development or small tasks like scripting.

  Python is particularly flexible because of the range of programming paradigms and styles it supports. The interactive interpreter (the REPL) makes experimentation easy and fun, and many people do serious work at the interpreter.

  New programmers can start with simple scripts, not even needing to write functions if they aren't necessary, moving up through procedural programming to full blown object oriented programming, functional programming and even metaprogramming. With first class namespaces, types and functions all these different programming paradigms are available to the Python programmer.

  Python has a rich standard library (it comes "batteries included") that makes it ready to use for a wide range of tasks straight out of the box. There is also a thriving community with a healthy ecosystem of third party libraries and extensions.

  The Python language and community extol readability and clarity as the highest virtues in programming. The indentation based block structure, which some people hate but which most people come to appreciate very quickly, is just one way that Python encourages an easy to read programming style.

• Comparing with other programming languages, what advantages and disadvantages does Python have?

  Python is a highly expressive, dynamically typed language. It is much more productive to use Python than a lower level systems programming language. As one example the codebase of Mercurial (a distributed version control system written in Python) is about 1/10th the size of the git codebase (with comparable features but written in C, Perl and shell scripts). Even when compared to statically typed languages like Java and C#, Python's type system and language features mean that development will typically be faster and result in a smaller code base.

  The trade-off is that dynamically typed languages tend to be slower than statically typed languages. On the other hand if you complete the initial implementation faster then you have more time to work on performance optimisation. This really has been my experience - there is much more to be gained in optimising your algorithms than there is from using a different language.

  Those who are used to statically typed languages may mourn the passing of type safety in dynamically typed languages, but really type safety is an extremely thin layer of safety. Just because a program compiles doesn't tell you anything about the correctness of the application logic for example. (Otherwise there would never be any bugs in applications written in statically typed languages.) The only way to be serious about the correctness of code is extensive testing (personally I'm an advocate of Test Driven Development, whichever language you are developing in). It just so happens that dynamically typed languages are particularly easy to test because you can create and modify types at runtime and don't have to fight against the compiler. There are many powerful testing libraries and tools available for Python.

• How do you comment of Python's rate of usage by people?

  Python is very widely used, and has risen in popularity a great deal in the last few years. I think in part the growth of Python recently can be attributed to the publicity around Ruby and Ruby on Rails. Suddenly dynamic languages have become not just acceptable but actually trendy and exciting.

  However Python use has been growing in many different areas, not just web development.

  Since I first started using Python in 2002 it has become virtually the de-facto administration language for Linux. Pardus is one example of this, Ubuntu is another, where both distributions do everything they can with Python. The Gentoo packaging system, portage, is written in Python and there are plenty of other examples.

  Python gets used for science and research, both because it is easy for scientists (who are not primarily programmers) to learn and use and also because of powerful libraries (often wrapping libraries written in C, C++ or Fortran) like Numpy, Scipy, BioPython, matplotlib, NLTK (Natural Language Toolkit) and so on.

  Python is also the standard embedded in both the GIS (Geographic Information Systems) and CGI (Computer Generated Imagery) worlds. Large animation houses like Pixar, Imageworks and Industrial Light & Magic all use Python. Applications like Blender and Maya embed Python for scripting.

  Python is used for hardware automation, testing systems, application development, web development and just about anything else you can think of. Python is a great general purpose programming language.

• Have you ever think to develop an official compiler to create binary files? Or does Python have already?

  Well, Python is a highly dynamic language so needs the runtime. It is already a bytecode compiled language but is generally described as an interpreted language because the runtime interprets the bytecode.

  Other bytecode compiled languages, like Java and C#, are usually described as compiled languages because they have a Just-In-Time compiler (JIT) that generates and executes native code instead of interpreting bytecode.

  There are a few projects to bring a JIT to Python. The most ambitious one is PyPy, an interpreter compiler toolchain that can compile dynamic language interpreters from source code in a static-subset of Python called RPython (Restricted Python). The project also includes a full implementation of the Python language in RPython (plus a smalltalk and prologic implementations). It has native, IL (Intermediate Language - for execution on the .NET and Mono implementations of the Common Language Infrastructure specification by Microsoft) and JVM (Java Virtual Machine) backends - so that a single implementation of a language can be used on all three platforms.

  The exciting, and slightly crazy, part of the project is that it also includes a JIT compiler. The PyPy toolchain generates a tracing JIT for language interpreters it compiles. For a long while this was hypothetical and experimental but after several years work there are now lots of benchmarks that run faster on PyPy than they do on CPython (CPython is the standard implementation of Python written in C).

  Note

  This interview was written before the announcement that Unladen Swallow is aiming to merge with Python 3.

  Another project is Unladen-Swallow, sponsored by Google who use Python a lot and have a commercial interest in seeing it run faster. Unladen Swallow uses the LLVM (Low Level Virtual Machine) project to add a JIT to Python. The project includes at least two Python core developers and they see it as a branch rather than a fork; their goal is to merge the performance enhancements back into core Python. The latest releases of Unladen Swallow do show speed improvements over CPython. LLVM is written in C++, and is not brilliantly tested on Windows, both of which could be 'issues' for merging back into Python - so we'll see...

  There is also Psyco, which is a specializing compiler that can be bolted-on to CPython. For numeric code in particular it offers massive speed improvements. That project stagnated for a long time, just about maintained but not being developed, but has recently seen a lot of improvements thanks to the work of Christian Tismer.

  There are other approaches to creating binary distributions of applications. These typically involve bundling the Python runtime and all the libraries you use together. On Windows a tool called py2exe will do this for you and on the Mac there is py2app. For Linux there is a tool called cx_Freeze and at least one other whose name escapes me at the moment.

  Another approach to creating binary applications written in Python is to use a hybrid language like Cython, Pyrex or Shedskin. Shedskin is a static subset of Python that compiles to C++. If you can live without some of the dynamic features of Python you get a compatible syntax, a big performance boost, and compiled binaries. Cython and Pyrex are more often used for writing Python extensions. They are a combination of the Python language with C. You can use Python syntax and semantics mixed and matched with C where you need the speed.

• What advantages Python 3 have comparing with Python 2?

  The big advantage is that strings and bytes are fully disentangled. In Python 2 we have two string types, the bytestring and Unicode strings. Although it is possible to handle text 'properly' with Python 2 you have to be strict about decoding and encoding at the application boundaries and it is much easier to do-the-wrong-thing and treat text as binary data. If you mix the approaches in Python 2 (sometimes using Unicode and sometimes treating text as binary data) then you're in for a world of hurt (encoding errors in odd places). Python 3 cleans all this up and all strings are Unicode. There is a new bytes type for when you are dealing with binary data. There is a new IO layer in Python 3 to support this.

  This alone makes it worth switching to Python 3 in my opinion, and this change couldn't be made without breaking backwards compatibility as it is such a fundamental change.

  There are lots of other language changes, there is now only one integer type (unification of the int and long types), comparisons and sorting can only be done with compatible types (the arbitrary ordering of different types in Python 2 didn't really make sense), lots of built-in methods return iterators instead of lists and so on.

  The new nonlocal statement is one of my favourite additions in Python 3. It adds true lexical scoping to Python (for changing variables in outer scopes - Python 2 has had lexical scoping for accessing values for a long time). There are also interesting features like function annotations that allow for attaching extra information to function / method signatures. This could be used for enforcing type contracts for example. Programmers are only just beginning to explore some of the things Python 3 makes possible.

  The main advantage Python 3 has is that it is the future. Python 2 won't be around for ever and new features and libraries will be added to Python 3 that are never available in Python 2.

• It's known that Python 3 doesn't support some of Python 2 codes? Is it really needed?

  Unfortunately yes. Some of the changes, particularly the string / bytes changes, just couldn't be introduced in a backwards compatible way. Honestly if it had been possible it would have been done.

• What important changes wait for us in future versions of Pyton 2 and Python 3?

  One of my favourite changes are all the improvements to the unittest library that come in 2.7 / 3.2.

  In Python 2.7 we are trying to restrict new features to ones that are already present in Python 3. This will make the transition between Python 2 and Python 3 easier. In Python 2.7 this means, for example, the addition of the ordered dictionary to the standard library that was added in Python 3.1.

  Because of the language moratorium there are very few language changes coming in Python 3.2. There are plenty of other improvements though, including a big improvement to the way floating point numbers are represented (by Mark Dickinson), substantial performance improvements to the Global Interpreter Lock (by Antoine Pitrou), the with statement works with multiple context managers, many bug fixes and so on.

  For future development I'd like to see a focus on improving the standard library. In particular although the whole standard library has been converted to work with Python 3 (at least to the point where all the tests pass), a lot of code that previously worked with strings may need careful auditing to see whether it would be better off working with bytes.

• When is the end of life time of Python 2?

  It looks likely that Python 2.7 will be the last major version of Python. This isn't set in stone, but maintaining the two branches of Python in parallel is a strain on the developers and most of them would rather just be working on Python 3.

  (It isn't just two branches - we have Python 2.6 being maintained and Python 2.7 being developed, plus Python 3.1 being maintained and 3.2 being developed. That means bug fixes currently have to be pushed to four branches.)

  Even if Python 2.7 is the last major release it will be maintained for several years to come, but the end of life is in sight.

• Which GUI (Qt, GTK, etc.) do you prefer while programming with Python?

  I've used both Tkinter (bundled as part of the Python standard library) and wxPython in the past. Qt has a very good reputation but the license for using PyQt on Windows was until recently only available under the GPL license or a commercial license. That made it a difficult choice for cross-platform development.

  wxPython is ok. It is greatly improved using it from Python with a wrapper library like Dabo that gives it a more Python-like feel.

  To be honest my recent GUI development work has all been with Windows Forms for Windows, using IronPython. Surprisingly enough this has a reasonably good API and looks very good on Windows. The Mono implementation of Windows Forms is extensive, but it looks pretty bad by default on the Mac (my main development machine is a Mac). I've seen some very good looking Windows Forms applications run on the Mac with Mono, so I know it's possible I just never learned the right magic tricks.

  If the cross-platform license situation improves with PyQt then I would be tempted to use that for future projects. It looks like this is happening in the form of the PySide project, sponsored by Nokia who own the Qt library.

• What about Tkinter? Is there any plan about developing Tkinter?

  Tkinter looks pretty quirky! It also feels quirky to develop for, but if you just need something simple it is nice to use something that is already in the standard library.

  Tkinter is based on Tk, part of the Tcl language. Tk itself has moved on a lot and in Tcl 8.5(?) a fancy new user-interface layer called Ttk became the standard. This is much better looking and has been incorporated into Tkinter in Python 2.7 and 3.2 as the Ttk module.

  I've seen screenshots of some very good looking applications written with Ttk, but I have no idea how much magic you have to use to get attractive, native looking UIs or whether you get them for 'free'.

• What do you think about Free Software?

  I'm a huge fan of free software and very committed to the open source movement. I'm not an enemy of commercial software though. I think it is a good thing that programmers are able to earn a living selling programs. I don't see commercial software and free software as enemies, they can be very good for each other. The development of commercial software, and the money it brings in, sponsors a lot of open source development.

• If you use any GNU/Linux distribution, could you say what it is?

  I've used Linux on servers regularly. Most servers I've worked with have tended to be Debian based.

• Have you ever heard of Pardus GNU/Linux?

  I've certainly heard of Pardus, but mainly because of your use and support of Python.

• It's known that Python's license is compatible with GPL. Why do you need that?

  Well... personally that doesn't make much difference to me. I like, and tend to use for my own projects, the BSD license as they are "more free" than the GPL. On the other hand it would be a major downer if the Python license was incompatible with the GPL and GPL projects couldn't use Python. From that perspective it is important that the Python license is compatible with the GPL.

• What's the meaning of developing open source implementation of the Python programming language which is tightly integrated with the .NET Framework according to you?

  I see IronPython as a great way of getting developers currently using the Microsoft .NET ecosystem into using free software. IronPython, despite being developed by Microsoft, is fully free software. It's a small step from using IronPython to using Python.

• Opposing closed code softwares, free/open source alternatives are written. Wouldn't it be good if an open-source or free alternative of .NET is developed instead of developing IronPython?

  There is a very good free implementation of .NET called Mono - and IronPython runs on that too. IronPython is developed by Microsoft and so in no way does the development of IronPython take away from work on Mono.

• What's your favorite free software written in Python?

  Ha, actually quite a difficult question. Not many of the applications I use regularly are written in Python - except for a couple that aren't free software (the Wing IDE and the Resolver One spreadsheet application). Django is my current favourite free software library written in Python. Developing web applications with Django is great fun.

• What do you want to say our readers?

  Thank you for the opportunity to speak to you, and thank you for reading what I have to say.

  I've recently become involved in a Python related podcast called "A Little Bit of Python". Definitely worth listening to if you're interested in Python.

A Little Bit of Python Episode 3

A Little Bit of Python is an occasional podcast on Python related topics with myself, Brett Cannon, Jesse Noller, Steve Holden and Andrew Kuchling. We still don't have our own website although that is due to land any day now. Meanwhile episode 3 has just gone live. The topics covered include the Python transition to using Mercurial, the release of the first alphas of Python 2.7 and the furore caused by comments on the Python Package Index.

• Episode 3: A Little Bit of Python (mp3)
• Episode 3: A Little Bit of Python (m4a)

General links for the podcast feeds and a webpage with an embedded flash player:

• A Little Bit of Python mp3 rss feed
• A Little Bit of Python m4a rss feed
• Temporary webpage home for A Little Bit of Python

If you have feedback, insults or suggestions for new topics you can email us on: all@bitofpython.com.

We don't yet have the podcast listed on iTunes; we'll set that up as soon as we have a permanent home for the podcast.

Psyco 2 Binaries for Windows and Python 2.4, 2.5 and 2.6

Pysco is a specializing compiler (a kind of JIT) for Python written by Armin Rigo. The difficulty of maintaining and extending psyco was one of the motivating factors behind the inception of PyPy. Psyco itself was maintained to remain compatible with recent versions of Python but still didn't optimise more recent features like Python generators and floats.

Development of psyco was recently taken over by Christian Tismer. Christian made many improvements to Psyco, but Windows binaries were never released. Available here are compiled binaries of psyco 2 for Windows for Python 2.4, 2.5 and 2.6.

• pysco 2.0.0 for 32bit Windows and Python 2.6 (.zip)
• pysco 2.0.0 for 32bit Windows and Python 2.5 (.zip)
• pysco 2.0.0 for 32bit Windows and Python 2.4 (.zip)

(Compiled from SVN head on January 11th 2010.)

From the official psyco project page on Sourceforge, prebuilt binaries are only available for Python 2.5. I've also built binaries for psyco 1.6 for Python 2.4 and 2.6:

• pysco 1.6 for 32bit Windows and Python 2.6 (.zip)
• pysco 1.6 for 32bit Windows and Python 2.4 (.zip)

Binaries for Python 2.2 and 2.3 (pysco 1.5.1) are available from my Python modules page.

• Front page
• All By Date
• All By Category
• Blog Statistics
• RSS Feed
• Return to Top

Technorati...
This work is licensed under a Creative Commons Attribution-Share Alike 2.0 License.

Counter...

• IronPython in Action

  

  IronPython in Action is the first English book on IronPython. It is for Python and .NET programmers interested in making the most of Python on the .NET framework and Mono.

• Search Site

  	Search this Site:
  	www voidspace.org.uk

• Browsing

  Home > Python Index > Techie Blog

• Blogads

  Blogads

• Michael Foord

  • Contact Michael
  • Michael on Twitter
  • My bookmarks
  • Shared articles
  • Photos on Flickr
  • Voidspace Amazon Store
  • Amazon Computers: Todays Deals

• Hosting