Python Programming, news on the Voidspace Python Projects and all things techie.
Porting mock to Python 3
One of the nice new features in mock 0.7 is that it works with both Python 2 & 3. The mock module itself, even with all the freshly added docstrings, weighs in at less than 800 lines of code so compatibility is maintained with a single source base rather than the more recommended 2to3 approach. There are however about 1500 lines of test code that also need to work under Python 3; so whilst not a particularly difficult exercise it was not entirely trivial to get all the tests passing under Python 2.4, 2.5, 2.6, 2.7 and 3.2. Good tests make it much easier to have confidence that the port works. Attempting this without tests would be much more painful, even though it means there is more code to port.
I've written up all the changes needed for mock to support Python 3:
The all new and improved unittest: unittest2
At PyCon I gave a talk on some of the new features that are coming to unittest.
unittest is the Python standard library testing framework. In recent years unittest has languished whilst other Python testing frameworks like nose and py.test have innovated. Some of the best innovations have made their way into unittest which has had quite a renovation. In Python 2.7 & 3.2 a whole bunch of improvements to unittest will arrive.
The video of my talk (30 minutes) is up online:
I've turned the slides of the talk into an article:
The talk / article introduces unittest2, a back-port of the features in unittest to work with Python 2.4, 2.5 & 2.6. unittest2 is developed with a Mercurial repository on hg.python.org/unittest2 . unittest2 is already in use for the development of distutils2 and there is an open ticket to make use of it in Django.
The new features include:
- automatic test discovery and new command line features
- addCleanups - better resource management
- many new assert methods including better defaults for comparing lists, sets, dicts unicode strings etc and the ability to specify new default methods for comparing specific types
- assertRaises as context manager, with access to the exception object afterwards
- test skipping and expected failures
- load_tests protocol for controlling the loading of tests from modules or packages
- startTestRun and stopTestRun methods on TestResult
- various other API improvements and fixes
You can install unittest2 with:
pip install unittest2
To use it in your modules change import unittest to import unittest2. Alternatively you can conditionally make use of it where it is available:
try: import unittest2 as unittest except ImportError: import unittest
|||Thanks to Dirkjan Ochtman for setting this up and finally forcing me to make the jump to Mercurial. I've found the switch very easy and am now a big fan of Hg.|
Psyco 2 Binaries for Windows and Python 2.4, 2.5 and 2.6
Pysco is a specializing compiler (a kind of JIT) for Python written by Armin Rigo. The difficulty of maintaining and extending psyco was one of the motivating factors behind the inception of PyPy. Psyco itself was maintained to remain compatible with recent versions of Python but still didn't optimise more recent features like Python generators and floats.
Development of psyco was recently taken over by Christian Tismer. Christian made many improvements to Psyco, but Windows binaries were never released. Available here are compiled binaries of psyco 2 for Windows for Python 2.4, 2.5 and 2.6.
- pysco 2.0.0 for 32bit Windows and Python 2.6 (.zip)
- pysco 2.0.0 for 32bit Windows and Python 2.5 (.zip)
- pysco 2.0.0 for 32bit Windows and Python 2.4 (.zip)
(Compiled from SVN head on January 11th 2010.)
From the official psyco project page on Sourceforge, prebuilt binaries are only available for Python 2.5. I've also built binaries for psyco 1.6 for Python 2.4 and 2.6:
Binaries for Python 2.2 and 2.3 (pysco 1.5.1) are available from my Python modules page.
PyCrypto 2.1.0 Binaries for Windows 32bit Python 2.6, 2.5 and 2.4
PyCrypto is a Python cryptography package originally created by Andrew Kuchling and now maintained by Dwayne C. Litzenberger. For a while I've been hosting Windows binaries for version 2.0.1. Dwayne has now done a new release, version 2.1.0.
I've built installers for 32 bit Windows for Python 2.6, 2.5 and 2.4. The 2.6 installer was built with Visual Studio 2008. The Python 2.5 and 2.4 installers were built with Visual Studio .NET 2003. They were built without GMP support (only needed for Crypto.PublicKey._fastmath). The installers come without guarantees, but all the tests pass.
- PyCrypto 2.1.0 for Python 2.6 (.zip)
- PyCrypto 2.1.0 for Python 2.5 (.zip)
- PyCrypto 2.1.0 for Python 2.4 (.zip)
- PyCrypto 2.1.0 Announcement and Release Notes
- PyCrypto Homepage
- Windows Binaries Download (for 2.0.1 & 2.1)
New IronPython Articles: Introduction to IronPython, Python for .NET Programmers, Dark Corners and Metaprogramming
I've put up three new articles in the IronPython section of this website. The first two are aimed at developers new to IronPython, particularly .NET developers. The third article should be useful to anyone using or interested in IronPython:
Just getting started with IronPython or wondering why you should be interested? This article introduces IronPython, explains how it came into existence and some of its more common use cases. The article also shows the basic IronPython and .NET integration with the interactive interpreter. The IronPython interactive interpreter is a great way to learn plus a useful tool for working with the .NET framework. The article even shows how the Dynamic Language Runtime allows languages to interoperate by using a Ruby library (through IronRuby) from IronPython.
An introduction to the Python programming language for programmers interested in IronPython. It covers the basic syntax and semantics and touches on how dynamic languages are better (oops - I mean different) to statically typed languages.
The IronPython team have done a very good job of integrating Python and the .NET framework without having to change Python or introduce new syntax. Despite this there are various areas where you need to know some IronPython specific information; previous experience of Python or C# alone will not be enough.
This article contains a lot of the nitty gritty details of how to use Python the language to integrate with the underlying .NET framework.
A fourth article of mine has also gone online, although without any action - or even knowledge - on my part. An extract from chapter 8 of IronPython in Action has been posted to Developer Zone. This is the section on metaprogramming, introducing and explaining metaclasses in Python:
Try Python: Python Tutorial and Interpreter in the Browser
Ever since Silverlight, with the capability of running Python code in the browser, I've wanted to create something to help people learn Python.
Try Python is the Python tutorial, with an interactive Python interpreter, running in the browser. It runs on Safari, Firefox and IE on Windows and the Mac and needs Silverlight 2 or 3 installed. At the moment it doesn't work with Moonlight 2 due to a bug in the beta, but hopefully it will work with the final release meaning you will be able to use Try Python with Linux and Firefox.
- Try Python: Online Tutorial and Interpreter in the Browser
- Google Code: Project source code, repository and issue tracker
Every code example has a button that allows you to execute it the interpreter that is on the right of the interface:
Try Python has the following features:
- Auto-resizes with the browser
- Navigation bar through the tutorial pages at the top and bottom
- Individual pages are bookmarkable and the URL fragment auto-updates when you change page
- Mouse scroll wheel supported over the tutorial and console scroll viewers
- Control-C interrupts the running code with a keyboard interrupt
- raw_input and input work in the interpreter
- Basic auto-indent and auto-dedent in the console
- Console history, including multiline history
- Syntax highlighting in the console
- reset command to clear the console
- Assign to sys.ps1 and sys.ps2 from the console
Not much of the Python standard library is included. I intend to expand the tutorial adding new modules as they are needed (the whole standard library is about 5mb and would make Try Python take much longer to load).
Some of the console history code was contributed by Resolver Systems.
There are various possible ways forward (there are some ideas on the issues list). I'm about one quarter of the way through an implementation of file and open that uses local browser storage - so users can follow the part of the tutorial that does file I/O. A simple text editor (and import hook) that lets you create and import modules and packages in local browser storage would also be good.
More tests are also needed. You can see the start of an in browser test framework at: unittest in the browser.
I think what I'd like to do is something more interactive though - that presents short snippets of code with explanation and lets you work through them one at a time. This would be more interactive than the page by page style at the moment. Ideas and contributions welcomed of course...
Introduction to Testing with unittest
I've put a new article on my website, it's an introduction to testing in Python with the standard library test framework, unittest:
The article is an introduction to the basic API and classes of unittest, from creating individual test modules to setting up a testing framework. It covers the different assert methods provided by the unittest test fixture and also topics like automatic test discovery, and testing techniques from mock objects to monkey patching to dependency injection.
- urllib2 the Missing Manual
- Basic Authentication
- Metaclasses in Five Minutes
- Basic Hints for Windows Command Line Programming
- OOP with Python
- An Introduction to ConfigObj
IronPython Tools and IDEs
A frequent question on the IronPython mailing list is "what IDE should I use with IronPython?". For many .NET developers the question is phrased slightly differently, "how do I use IronPython in Visual Studio?". There are now several different major IDEs with IronPython support, including a few different ways of using IronPython with Visual Studio.
I've written a roundup of the major editors and how they support IronPython. This includes a look at the standard features you would expect in a Python editor, like autocomplete, calltips, debugging and more - with honourable mentions for other Python editors like Vim, Emacs, Komodo, Davy's IronPython Editor and the DLR editor that comes with the Pyjamas project. The article also has a roundup of standard tools for Python development; the code quality tools (PyLint, PyChecker and PyFlakes), profilers, debuggers, coverage, refactoring and so on.
- IronPython Studio
- Visual Studio
- Visual Studio SDK Experimental Hive
- Wing IDE
- Eclipse and PyDev
- Other tools
- Windbg SOS
- Code Quality Checkers
- Debugging and the Python Pdb Module
- Code Coverage and Profiling
- Refactoring and Complexity
Writing a Technical Book
Any day now I will get the first quarter sales figures for IronPython in Action. That will mark the book having been actually in the hands of readers for three months and also be about two and a half years since I first contacted Manning about the possibility of writing an IronPython book for them.
I've written up my experiences of writing a technical book, including my justification (uh, I mean explanation) of why it took so long, the writing process, things I learned and contract advice for the aspiring writer. This is advice I didn't follow myself but wish I had...
Much of the article is specific to my experiences with IronPython in Action - but despite the difficulties (and there will always be difficulties) I still recommend Manning if you really have to write a technical book:
The most important thing I learned was don't sweat the small stuff. This warrants repeating. Don't sweat the small stuff. Many times I knew the gist of what I wanted to say in a passage but couldn't find the words. I would go round and round over a single sentence for fifteen minutes or more. This happened a lot. I learned to just write something and then come back to it later. Often what I had been unhappy about when writing read fine when I came back to it the next day. If I was really stuck I would just leave a placeholder (like XXXX or something easy to search for) and come back to it another time. Letting yourself get stuck drags out the writing process and makes it mentally exhausting. Far better to just write what you have and move on; you're going to be going back over it later anyway.
Catching up: Pythonutils 0.4.0, akismet 0.2.0 and article updates
About two and a half years ago I started writing the book. During the next two years I received a steady trickle of feature requests, bug reports and patches for the various projects and articles I maintain (or pretend to maintain). For the most part I stuck these emails in an 'outstanding' folder promising to deal with them when the book was done (which at the time seemed like it would never happen).
IronPython in Action was properly finished at the beginning of the year (actual writing finished a while before that) and available in the shops nearly 3 months ago; I should be getting sales figures for the first quarter any day now. I still owe you a blog entry on the experience of writing a technical book for Manning. (Executive summary: it isn't such a hot idea whilst working full time and commuting four hours a day.)
Well, incredibly, I've been working through my backlog. I started with 138 emails in my outstanding folder - which included several duplicates and an almost entire Python-dev thread on unittest so not as many as it sounds - and now I'm down to 21 emails. Here are some of the things I've been working on:
akismet.py is a module for accessing the Akismet anti-spam web service. Useful for blogs or applications which accept user comments and want to check for spam.
This release (0.2.0) adds compatibility with Google AppEngine.
All changes in 0.2.0:
- If the data dictionary passed to comment_check doesn't have a 'blog' entry it will be added even if build_data is False. Thanks to Mark Walling.
- Fix for compatibility with Google AppEngine. Thanks to Matt King.
- Added a setup.py - install with pip or easy_install
This package is a collection of general Python utility modules, mostly old now. The old modules are not being actively developed and are in bugfix only maintenance mode. The main reason for me doing a new release is to stop people reporting the same bug in pathutils over and over again...
Code corrections pointed out by Davy Mitchell
Some time ago Python basic authentication handling changed to require the use of the protocol in urls passed to HTTPPasswordMgrWithDefaultRealm().add_password. The article has been updated to reflect this.
Addition of notes on BadStatusLine and HttpException Exceptions.
HOWTO: Using the Wing Python IDE with IronPython
A common question amongst new IronPython users is Which IDE is best for IronPython?
One common suggestion, which .NET developers gravitate towards naturally, is IronPython Studio. This is an example of extending Visual Studio through the VSx shell, and in my opinion not really suitable for pain-free use. You can read some of my thoughts on it in this IronPython-URLs blog entry.
As IronPython code is just Python code any good Python IDE will do, however they rarely feature good integration with IronPython. Fortunately there are a large range of tools that can be plugged into any extensible editor.
My favourite IDE is the Wing IDE, not least because of it has the best autocomplete (intellisense) of any Python IDE I've used. It achieves this by statically analysing Python code and inferring the types. This doesn't work with .NET types because they don't have Python source code... This HOWTO shows 'how to' enable autocomplete for the .NET types in Wing, plus using the scripting API to add commands like executing the current file with IronPython:
This work is licensed under a Creative Commons Attribution-Share Alike 2.0 License.