Voidspace

unittest2 0.2.0 Released and News on nose

I've pushed out a new release of unittest2, version 0.2.0.

unittest2 is a backport of all the fancy new features added to the unittest testing framework in Python 2.7. It is tested to run on Python 2.4 - 2.6.

• unittest2 Mercurial Repository
• Article on new features in unittest
• unittest2 on PyPI

The biggest new feature in the 0.2.0 release is the addition of shared fixtures; setUpClass, setUpModule, tearDownClass and tearDownModule.

Before I explain these new features there is some interesting news about the next version of nose. Jason Pellerin, creator and maintainer of nose, sent an email to the Testing in Python mailing list. unittest2 and the future of nose:

nose has always been intended to be an extension of unittest -- one that opens unittest up, makes it easier to use, and easier to adapt to the needs of your crazy package or environment.

Fact: unittest2 is coming: http://www.voidspace.org.uk/python/articles/unittest2.shtml

... indeed, it's already here (just not yet in stdlib). It does test discovery (though not by default), can support test functions (though not by default), will eventually support a better form of parameterized tests than nose does, and also will eventually support at least class and module-level fixtures.

Second fact: I have very little time to work on nose anymore, and with offspring #2 due any day now, that's not going to change much for at least a few years.

Third fact: setuptools is dying, distribute is going with it, and something called distutils2 is going to rise to take its place. This matters because a LOT of nose's internal complexity is there to support one thing: 'python setup.py test'. If that command goes away or changes substantially, nose will have to change with it.

Fourth fact: many high-level nose users want it to be released under a more liberal license than LGPL and won't become contributors until/unless that happens.

I think this all adds up to starting a new project. Call it nose2. Call it @testinggoat. Whatever it's called, it should be based on extending unittest2, and designed to work with distutils2's test command. It should support as much of the current nose plugin api as is reasonable given the changes in unittest2. And it should be BSD or MIT licensed (or some reasonable equivalent) so more people feel comfortable contributing.

Actually Jason is mistaken about a couple of the details, unittest2 is just a backport of what is already in the Python standard library and test discovery is supported right out of the box (not quite sure what he means by 'default'). That aside, nose has inspired a lot of the improvements that are in unittest2, and hose still has plenty of features that have not yet made it across. There are some features from nose that will never make it into core unittest, but the foundation of unittest2 and distutils2 will make a new version of nose substantially simpler.

Various people have volunteered to work on the shiny new nose (bite my shiny metal nose?), but the first step is for unittest to gain a decent extension API. After Python 2.7 hits beta this is my highest priority for unittest. Follow the rest of the nose2 email thread for an idea of how it all fits together.

Class and Module Level Fixtures

unittest2 now allows you to create class and module level fixtures; these are versions of setUp and tearDown that are run once per class or module.

Class and module level fixtures are implemented in TestSuite. When the test suite encounters a test from a new class then tearDownClass from the previous class (if there is one) is called, followed by setUpClass from the new class.

Similarly if a test is from a different module from the previous test then tearDownModule from the previous module is run, followed by setUpModule from the new module.

After all the tests in the suite have run the final tearDownClass and tearDownModule are run.

setUpClass and tearDownClass

These must be implemented as class methods.

import unittest2

class Test(unittest2.TestCase):
    @classmethod
    def setUpClass(cls):
        cls._connection = createExpensiveConnectionObject()

    @classmethod
    def tearDownClass(cls):
        cls._connection.destroy()

If you want the setUpClass and tearDownClass on base classes called then you must call up to them yourself. The implementations in TestCase are empty.

If an exception is raised during a setUpClass then the tests in the class are not run and the tearDownClass is not run. Skipped classes will not have setUpClass or tearDownClass run.

setUpModule and tearDownModule

These should be implemented as functions.

def setUpModule():
    createConnection()

def tearDownModule():
    closeConnection()

If an exception is raised in a setUpModule then none of the tests in the module will be run and the tearDownModule will not be run.

There are a few important details about these functions, particularly how they behave when test suites are randomized. More information in the docs.

The all new and improved unittest: unittest2

At PyCon I gave a talk on some of the new features that are coming to unittest.

unittest is the Python standard library testing framework. In recent years unittest has languished whilst other Python testing frameworks like nose and py.test have innovated. Some of the best innovations have made their way into unittest which has had quite a renovation. In Python 2.7 & 3.2 a whole bunch of improvements to unittest will arrive.

The video of my talk (30 minutes) is up online:

• New *and* Improved: Coming changes to unittest

I've turned the slides of the talk into an article:

• Coming changes to unittest in Python 2.7 & 3.2

The talk / article introduces unittest2, a back-port of the features in unittest to work with Python 2.4, 2.5 & 2.6. unittest2 is developed with a Mercurial repository on hg.python.org/unittest2 [1]. unittest2 is already in use for the development of distutils2 and there is an open ticket to make use of it in Django.

• unittest2 on PyPI
• unittest2 mercurial repo

The new features include:

• automatic test discovery and new command line features
• addCleanups - better resource management
• many new assert methods including better defaults for comparing lists, sets, dicts unicode strings etc and the ability to specify new default methods for comparing specific types
• assertRaises as context manager, with access to the exception object afterwards
• test skipping and expected failures
• load_tests protocol for controlling the loading of tests from modules or packages
• startTestRun and stopTestRun methods on TestResult
• various other API improvements and fixes

You can install unittest2 with:

pip install unittest2

To use it in your modules change import unittest to import unittest2. Alternatively you can conditionally make use of it where it is available:

try:
    import unittest2 as unittest
except ImportError:
    import unittest

Please use unittest2 and report bugs, or make feature requests, on the unittest2 issue tracker or the Python issue tracker.

New Release: ConfigObj 4.7.2

A new version of ConfigObj has just been released: 4.7.2

This is a bugfix release with several important bugfixes. It is recommended that all users of ConfigObj 4.7 update.

• Direct Download Link
• PyPI Page
• ConfigObj Home Page & Documentation

The new version can be installed with:

pip install -U configobj

Changelog

• BUGFIX: Restore Python 2.3 compatibility
• BUGFIX: Members that were lists were being returned as copies due to interpolation introduced in 4.7. Lists are now only copies if interpolation changes a list member. (See below.)
• BUGFIX: pop now does interpolation in list values as well.
• BUGFIX: where interpolation matches a section name rather than a value it is ignored instead of raising an exception on fetching the item.
• BUGFIX: values that use interpolation to reference members that don't exist can now be repr'd.
• BUGFIX: Fix to avoid writing '\r\r\n' on Windows when write is given a file opened in text mode ('w').

See below for important details on the change to using list values with string interpolation.

What is ConfigObj?

ConfigObj is a simple but powerful config file reader and writer: an ini file round tripper. Its main feature is that it is very easy to use, with a straightforward programmer's interface and a simple syntax for config files. ConfigObj has a host of powerful features:

• Nested sections (subsections), to any level

• List values

• Multiple line values

• Full Unicode support

• String interpolation (substitution)

• Integrated with a powerful validation system

  • including automatic type checking/conversion
  • and allowing default values
  • repeated sections

• All comments in the file are preserved

• The order of keys/sections is preserved

• Powerful unrepr mode for storing/retrieving Python data-types

String Interpolation and List Values

For general information on string interpolation in ConfigObj see: String Interpolation in ConfigObj.

Since version 4.7 string interpolation is done on string members of list values. If interpolation changes any members of the list then what you get back is a copy of the list rather than the original list.

This makes fetching list values slightly slower when interpolation is on, it also means that if you mutate the list changes won't be reflected in the original list:

>>>  c  =  ConfigObj()
>>>  c['foo']  =  'boo'
>>>  c['bar']  =  ['%(foo)s']
>>>  c['bar']
['boo']
>>>  c['bar'].append('fish')
>>>  c['bar']
['boo']

Instead of mutating the list you must create a new list and reassign it.

dict.__getitem__(configobj, key)

ConfigObj 4.7.1 (and how to test warnings)

I hate doing releases. I haven't managed to automate the whole process (I should probably work on that), although setup.py sdist upload certainly helps. Anyway, the short version of the story (and the real reason I hate releases) is that was a bug in ConfigObj 4.7.0. 4.7.1 is a brown paper bag release to fix the bug

• ConfigObj 4.7.1 (.zip)
• ConfigObj Homepage
• ConfigObj on PyPI

The bug was an error in the way I had setup the deprecation warning for the obsolete options dictionary in the ConfigObj constructor. The bug only affects you if you were still using the options dictionary to configure ConfigObj instances.

If you've never heard of ConfigObj now is an ideal time to try it out. ConfigObj is a simple but powerful config file reader and writer: an ini file round tripper. Its main feature is that it is very easy to use, with a straightforward programmer's interface and a simple syntax for config files. It has lots of other features though :

• Nested sections (subsections), to any level

• List values

• Multiple line values

• Full Unicode support

• String interpolation (substitution)

• Integrated with a powerful validation system

  • including automatic type checking/conversion
  • and allowing default values
  • repeated sections

• All comments in the file are preserved

• The order of keys/sections is preserved

• Powerful unrepr mode for storing/retrieving Python data-types

The reason the bug happened is because not only did I not even try the deprecation warning to make sure it worked, let alone add a test for it. It turns out that adding a test for warnings is easy using the catch_warnings context manager, new in Python 2.6.

from warnings import catch_warnings
with catch_warnings(record=True) as log:
    ConfigObj(options={})

# unpack the only member of log
warning, = log
self.assertEqual(warning.category, DeprecationWarning)

Discover 0.3.2 and the load_tests protocol

discover is a test discovery module for the standard library unittest test framework. Test discovery is built into unittest in Python 2.7 and 3.2. The discover module is a back-port of test discovery to work with Python 2.4 - 2.6 and Python 3.0 / 3.1 [1].

3.2 is a minor bugfix release. Test discovery also includes a new protocol called load_tests. In previous versions the standard tests would be passed in as a list instead of a TestSuite instance. This bug is now fixed both in untitest and in discover.

• discover module on PyPI
• unittest-ext google code project (SVN)

discover can be installed with pip or easy_install. After installing switch the current directory to the top level directory of your project and run:

python -m discover
python discover.py

This will discover all tests (with certain restrictions) from the current directory. The discover module has several options to control its behavior (full usage options are displayed with python -m discover -h). See the documentation on the PyPI homepage for details.

The load_tests protocol is interesting. It allows you to customize how tests are loaded from a module by defining a load_tests function. load_tests takes three arguments, the test loader, the standard set of tests for that module (allowing you to just add tests to the standard set if you want). The third argument is only used for load_tests functions in the __init__.py of test packages.

Here's an example of a test module with two test classes. The load_tests function returns a test suite that only uses one of the test classes:

import unittest


class FirstTest(unittest.TestCase):
    def testFoo(self):
        self.fail()

class SecondTest(unittest.TestCase):
    def testFoo(self):
        pass

def load_tests(loader, tests, _):
    return loader.loadTestsFromTestCase(SecondTest)

When the test module is loaded by discover, or unittest from Python 2.7 / 3.2, then load_tests will be called to create the test suite for the module.

[1]	discover.py is only about 300 lines of Python. Supporting 2.x and 3.x in a single code-base is easy with small modules but I wouldn't recommend it for larger projects.

A Rambling Recording on Member Lookup in Python (podcast)

I was thinking about the Python object model, in part as a result of my post on The Python Class Statement. Python is a really easy language to learn, but it also has advanced features like its protocols, descriptors and metaclasses, that make the full object model pretty complex - and that's before you start looking at the corner cases.

It would be really nice to write up a single document describing the Python object model, including all of its intricacies. That sounds too much like hard work, so instead I recorded a rambling hand-wavy description of member lookup in Python. I don't go into full blown detail, but then this is a podcast - it won't seriously mislead you and no-one is going to use it as a reference guide...

• Python Attribute Lookup Part 1 on Audioboo
• Python Attribute Lookup Part 2 on Audioboo
• Python Attribute Lookup in full mp3 (9minutes 8MB)

This was recorded using the Blue Fire iPhone app whilst I was wandering around outside. I chopped out about half my pauses and coughing using Audacity, so if you think the quality is rough you should have heard the first version.

Topics covered include:

• Member lookup on instances and classes
• How the interpreter looks up protocol ('magic') methods
• __getattr__ and its mysterious cousin __getattribute__
• Descriptors, bound methods, properties and friends

In the podcast I mention the new technique I have for dynamically mocking magic methods. Magic methods, when they are called for you by the interpreter, are usually looked up directly on the class. Unfortunately Python is not entirely consistent, some magic methods are still looked up on the instance first before the class. This is gradually being fixed in Python (in 2.7 they pretty much all fixed), but the inconsistency is a pain for mocking the magic methods.

Mock now allows you to mock the magic methods by assigning an appropriate function, that takes self as the first argument, to the magic method on the mock instance. By default mocks do not have the magic methods implemented except the ones it uses itself. When you assign to them it dynamically grows them on just that instance - all other mock instances are unaffected. Magic methods can then be looked up on the class or the instance, either way works (and you can delete them):

>>> from mock import Mock
>>> m = Mock()
>>> m
<mock.Mock object at 0x429770>
>>> m.__repr__ = lambda self: 'A Mock Object'
>>> m
A Mock Object
>>> m.__repr__()
'A Mock Object'
>>> del m.__repr__
>>> m
<mock.Mock object at 0x429770>

You can also use Mocks for magic methods. Here's an example of mocking out the built-in open function when used as a context manager:

@patch('__builtin__.open')
def test_with_statement(self, mock_open):
    mock_open.__enter__ = Mock()
    mock_open.__exit__ = Mock()
    mock_open.__exit__.return_value = False

    with open('filename') as handle:
        handle.read()

    mock_open.assert_called_with('filename')
    mock.__enter__.assert_called_with()
    mock.__enter__.return_value.read.assert_called_with()
    mock.__exit__.assert_called_with(None, None, None)

The version of mock with magic method support hasn't yet been released, but you can pull it from the google code SVN repo. When I have time to write docs it will be released as 0.7.0.

There's a bit of trickery involved in making this work. If you're interested in how it's done look at the implementation of __new__ and __setattr__.

Release: ConfigObj 4.7.0 and validate 1.0.1

I've just released ConfigObj 4.7.0 and validate 1.0.1. ConfigObj is an easy to use configuration file reader and writer module. ConfigObj is easy to use, with a host of powerful features including:

• Nested sections (subsections), to any level

• List values

• Multiple line values

• String interpolation (substitution)

• Integrated with a powerful validation system

  • including automatic type checking/conversion
  • repeated sections
  • and allowing default values

• When writing out config files, ConfigObj preserves all comments and the order of members and sections

• Many useful methods and options for working with configuration files (like the 'reload' method)

• Full Unicode support

The headline feature of 4.7.0 is an ~25% performance improvement in reading and validating configuration files. The cost of this performance improvement is losing compatibility with Python 2.2 [1]. Python 2.3 is now the minimum version of Python supported by ConfigObj.

The next new feature is the addition of the extra_values section attribute and the get_extra_values function. extra_values is populated by validation and lists all config file members (in that section) that weren't specified in the configspec file. get_extra_values is a convenience function that returns a list of all values and sections that appear in the config file but aren't in the configspec.

Another important change is that passing in arguments to the ConfigObj constructor via an options dictionary is not possible any more. This was a hangover from previous versions and if you were using it you can just change you code to the following (that works with older versions of ConfigObj as well):

config = ConfigObj(filename, **options)

4.7.0 also brings in a minor change in syntax. Previously spurious commas in list values would be ignored (e.g. value = first, , second). They are now invalid syntax.

There have been several other minor bugfixes. See the list below for details.

All Changes in Version 4.7.0

• Minimum supported version of Python is now 2.3
• ~25% performance improvement thanks to Christian Heimes
• String interpolation now works in list value members
• After validation any additional entries not in the configspec are listed in the extra_values section member
• Addition of the get_extra_values function for finding all extra values in a validated ConfigObj instance
• Deprecated the use of the options dictionary in the ConfigObj constructor and added explicit keyword arguments instead. Use **options if you want to initialise a ConfigObj instance from a dictionary
• Constructing a ConfigObj from an existing ConfigObj instance now preserves the order of values and sections from the original instance in the new one
• BUGFIX: Checks that failed validation would not populate default_values and restore_default_value() wouldn't work for those entries
• BUGFIX: clear() now clears 'defaults'
• BUGFIX: empty values in list values were accidentally valid syntax. They now raise a ParseError. e.g. "value = 1, , 2"
• BUGFIX: Change to the result of a call to validate when preserve_errors is True. Previously sections where all values failed validation would return False for the section rather than preserving the errors. False will now only be returned for a section if it is missing
• Distribution includes version 1.0.1 of validate.py
• Removed __revision__ and __docformat__

The only change in version 1.0.1 of validate was removing a test whose behaviour was dependent on platform / Python version and was impossible to make pass consistently.

Current State of Unladen Swallow (Towards a Faster Python)

I'm helping to organise the Python Language Summit that precedes the PyCon Conference this year. One of the first topics we'll be discussing is "Python 3 adoption and tools and the Python language moratorium" which will include representatives of the major Python implementations telling us the current state of their implementation and their plans or progress for supporting Python 3.

I was fortunate today to exchange emails with Collin Winter, one of the core developers of Unladen Swallow on this topic. He gave me a sneak preview of what he will have to say at the summit.

By way of introduction, Unladen Swallow is a project sponsored by Google to speedup Python. In particular it uses the LLVM (Low Level Virtual Machine) to provide a JIT (Just in Time compiler). Google use Python a great deal and so have a direct commercial interest in making it faster. Shortly after the first release Unladen Swallow was put to work serving YouTube traffic. The Unladen Swallow team see the project as a branch of CPython (the standard and reference implementation of the Python programming language) and their goal is to merge their changes back into Python once it is complete.

Here is what Collin had to say:

Brief summary: Unladen Swallow is currently focused on the process of merging into CPython's 3.x line (PEPs, patches, etc). We've been focused on pushing all our necessary changes into upstream LLVM in time for LLVM's 2.7 release, so that CPython 3.x can be based on that release. We've met our goals of maintaining pure-Python and C extension compatibility while speeding up CPython, and have created a platform for future development that we believe can continue to yield increasing performance for years to come.

As for py3k support, Unladen Swallow is currently based on CPython 2.6.1. We will seek to merge our changes into Python's 3.x line exclusively (without backporting to 2.x), updating our patches as necessary to correct for the 2.x->3.y skew. Once merger is completed, Unladen Swallow will cease to exist as an independent project.

(Emphasis added by me.) I also asked Collin about whether the merge back into Python woould be difficult, in particular because LLVM hasn't been well tested on Windows in the past and the LLVM is also written in C++ whereas the rest of CPython is all C. This potentially has ABI compatibility issues:

Unladen currently builds and passes its tests on Windows (or did the last time I checked), so I don't think that will be an issue.

The addition of C++ to the codebase will be more contentious, but the C++ stuff is restricted to the JIT-facing internals; the rest of the implementation is still straight C. We hope that will mitigate any concerns. "Impact on CPython Development" is a fairly lengthy section of the merger PEP I'm writing.

So, it looks like Unladen Swallow already offers a speedup that the team are happy with. I know there have been some interesting improvements recently, like this one that inlines certain binary operations. Additionally the merge with CPython is going to happen 'soon' (i.e. a PEP is being written) and it is likely to happen on the Python 3 branch. Perhaps that will give the Python community an incentive to make the jump...

NOTE: Jesse Noller also has a blog entry on this topic: Unladen Swallow: Python 3's Best Feature.

New Year's Python Meme

This is the blog entry I had nearly finished when I started messing around with Mock on Saturday. Started by Tarek Ziade, five short questions on you and Python in 2009 (or in this case me and Python)...

1. What's the coolest Python application, framework or library you have discovered in 2009?

   One of the biggest things that happened in my life in 2009 was leaving Resolver Systems to become a freelance developer working for a German firm Comsulting.de. I'm still working with IronPython, but now developing web applications with Django on the server and using IronPython in Silverlight on the client. It's great fun and although I'd used both Silverlight and Django a bit previously I'm now using working with them full time.

2. What new programming technique did you learn in 2009?

   At Resolver Systems we all took responsibility for architectural decisions. It was a great team to be part of and a great way to work. In my current project I've largely been responsible for building the application myself, although my colleague who is an excellent designer has recently been able to join me in the coding. That means I've made the architectural decisions for the application. This has stretched me and structuring large applications is something I want to explore more.

3. What's the name of the open source project you contributed the most in 2009? What did you do?

   Well, in 2009 I became a Python core-developer and the maintainer of unittest. The work I've done on unittest is definitely the most valuable contribution to open-source in 2009. That reminds me, there are a bunch of open tickets with my name on that I really ought to be looking at instead of doing this...

   Other than that I worked on a bunch of little projects of my own.

   • Try Python

     This is probably the project I'm most proud of: a Python interpreter and tutorial that runs in the browser with Silverlight. As Silverlight comes from Microsoft, and last I heard was installed on around 30% of the world's browsers, Try Python isn't a runaway success but I think it is very cool. Moonlight is now out, so in theory the site could work on Linux machines - but there is an issue with the version of IronPython I use. Hopefully I'll get around to updating the site soon and will also add an IronPython tutorial to the Python tutorial.

   • ConfigObj

     A Python configuration file reader and writer that is easy to use but with about a gazillion extra features not found in ConfigParser. This is the most widely used code I've ever released, but I don't use it much myself these days. Thankfully it takes little maintaining, however I have done a bunch of work on version 4.7 which is just waiting for me to pull my finger out and release.

   • Mock

     A simple mocking library for testing Python code, that makes a great companion to unittest. In my day job I'm now focusing on integration testing and not doing much unit testing, so I don't use Mock as much as I used to and it hasn't got the attention it deserves. It was nice to finally add support for magic methods so that you can mock numeric types, containers and so on.

   I also wrote a lot of articles on IronPython and supporting example code to go with them.

4. What was the Python blog or website you read the most in 2009?

   Like many of the other folk who answered these questions, I tend to keep in touch with Python news through Planet Python (in fact this year I sort of became responsible for some of the administration of the Planet when I joined the Python webmaster team). I enjoy a lot of the bloggers on the Planet and find it invaluable for keeping up to date with the Python world. There really are a lot of great bloggers contributing to the Planet so I'm only going to call out one: Jacob Kaplan-Moss. A great blogger, both fun and on the ball technically. Of course like the rest of us he needs to pull his finger out and blog more often.

   In fact in 2009 I went old school and (re)discovered the joys of IRC. I'm often on #python-dev and various other Python related channels. Twitter is also still growing and I've had a lot of fun and learned a lot from the many tech folk I follow there.

5. What are the three top things you want to learn in 2010?

   I'd like to learn more about web programming, in particular I want to get deeper into Django (and perhaps Pinax) and properly learn Javascript. There are lots of programming languages I'd like to learn (C so I can contribute to CPython and just because it is everywhere, Haskell so I can get functional enlightenment [1], maybe F# so I can achieve the same thing but in a language that might actually be useful, Erlang because all the cool kids are doing it and it seems to have the most practical approach to concurrency of the 'modern' languages, Lisp to see what all the fuss is about and probably a load more languages). In reality I'll only learn a programming language that I actually need to use, so I think Javascript is the programming language I'm most likely to have the opportunity to really dive into. Although I've tinkered with Javascript (who hasn't) I haven't fully appreciated what it means to idiomatically program with a prototype based language so it is definitely of value.

   There are a huge number of libraries and frameworks I'd love to learn, including Twisted, multiprocessing and other web frameworks. I'd also like to do mobile application development either for the iPhone or Android. That would give me a reason to use another language, but I have to say that Objective-C is more appealing than Java.

   I doubt I'll find time to do any of this in 'hobby-time', so hopefully they'll come up in a work context.

Mocking Magic Methods and Preserving Function Signatures Whilst Mocking

So, I'm most of the way through one blog entry, my tax return is due, I have a PyCon talk to write and I have a release of ConfigObj [1] just waiting for me to finish updating the docs. Naturally then I should mess around implementing new features for Mock.

These particular features were inspired by an email from Mock user Juho Vepsalainen who had a particular problem with Mock. In case you aren't familiar with it, Mock is a simple mocking library for unit testing. Mock makes creating mock objects, and patching out implementations with mocks at runtime, trivially easy.

I've spent a chunk of time today implementing a module that extends Mock to add new features. Eventually they will become part of Mock itself, but that would require a new release and tedious things like writing documentation:

• extendmock module
• tests and examples for mocksignature
• tests and examples for magic methods

To implement a lot of functionality (mocking any class and recording how they are used), mocks are instances of the Mock class. This can be a problem for code that uses introspection to determine if something is a function or not, or introspects the function signature. If you mock a function or method it will be replaced with a callable object with the signature (*args, **kwargs). This also means that code which is called incorrectly won't raise an error, you will only catch this in your tests if you specifically check how the object is called (which you usually will because that's the point of mocking it out - but still).

A solution to all these problems is the mocksignature function. This takes a function (or method) and a mock object. It creates a wrapper function with the same signature as the function you pass in. When called this wrapper function calls the mock, so instead of directly patching a mock to replace a function or method you use the function returned by mocksignature. Code that introspects the function you are patching out will still work. Here's an example:

from mock import Mock, patch
from extendmock import mocksignature

from some_module import some_function
mock = Mock()

mock_function = mocksignature(some_function, mock)

@patch('some_module.some_function', mock_function)
def test():
    from some_module import some_function
    some_function('foo', 'bar', 'baz')

test()
mock.assert_called_with('foo', 'bar', 'baz')

To make it more convenient to use I will build support for mocksignature into the patch decorator.

You can also use mocksignature on instance methods:

from mock import Mock
from extendmock import mocksignature

class Something(object):
    def method(self, a, b):
        pass

s = Something()
mock = Mock()
mock_method = mocksignature(s.method, mock)
s.method = mock_method

s.method(3, 4)
mock.assert_called_with(3, 4)

A limitation of mocksignature is that all arguments are passed to the underlying mock by position. If there are default values they will be explicitly passed in. Keyword arguments are only collected if the function uses **kwargs. See the tests for more details. The important fact is that the function signature is unchanged:

import inspect
from extendmock import mocksignature
from mock import Mock

def f(a, b, c='foo', **kwargs):
    pass
mock = Mock()

new_function = mocksignature(f, mock)
assert inspect.getargspec(f) == inspect.getargs(new_function)

The limitation on keyword arguments sounds confusing (certainly the way I expressed it above), so it's easier to demonstrate in practise with the call_args attribute:

>>> from mock import Mock
>>> from extendmock import mocksignature
>>>
>>> mock = Mock()
>>>
>>> def f(a=None): pass
...
>>> f2 = mocksignature(f, mock)
>>> f2()
<mock.Mock object at 0x441d70>
>>> mock.call_args
((None,), {})
>>> mock.assert_called_with(None)
>>>

Even though we passed no arguments in, the argument with the default value (a) is called as if None was passed in explicitly. This affects the way you use assert_called_with when using Mock and mocksignature in concert. You can still use mocksignature with functions that collect args with *args and **kwargs:

>>> from extendmock import mocksignature
>>> from mock import Mock
>>>
>>> def f(*args, **kw): pass
...
>>> mock = Mock()
>>> mock.return_value = 3
>>> f2 = mocksignature(f, mock)
>>> f2(1, 'a', None, foo='fish', bar=1.0)
3
>>> mock.call_args
((1, 'a', None), {'foo': 'fish', 'bar': 1.0})
>>>

Another problem with Mock is that it currently doesn't support mocking out the Python protocol methods (like __len__, __getitem__ and so on). extendmock contains a new class that adds magic suport to Mock: MagicMock. Here's an example of how you use it:

from extendmock import MagicMock

mock = MagicMock()

_dict = {}
def getitem(self, name):
    return _dict[name]
def setitem(self, name, value):
    _dict[name] = value
def delitem(self, name):
    del _dict[name]

mock.__setitem__ = setitem
mock.__getitem__ = getitem
mock.__delitem__ = delitem

self.assertRaises(KeyError, lambda: mock['foo'])
mock['foo'] = 'bar'
self.assertEquals(_dict, {'foo': 'bar'})
self.assertEquals(mock['foo'], 'bar')
del mock['foo']
self.assertEquals(_dict, {})

You mock magic methods by assigning a function (or a mock object) to the mock instance. Magic methods are looked up on the object class by the Python interpreter. MagicMock has all the magic methods implemented in a way that checks for corresponding instance variables, with sensible behaviour if the instance variable doesn't exist. However, the presence of these magic methods on the class could break some duck-typing (if it checks for the presence or absence of these methods), so I would rather have MagicMock be a separate class instead of integrating this into the Mock class. On the other hand there is no reason why I can't move MagicMock into the mock module next time I do a release.

For all magic methods you mock in this way you have to include self in the function signature. I might change this at a future date, so be warned this an experimental implementation. Also note that calls to mocked magic methods aren't recorded in method_calls and don't use object wrapping - all things that may change in the future.

One reason that some users have been requesting magic method support is for mocking context managers. Unfortunately __enter__ and __exit__ are looked up differently from the other magic methods in Python 2.5 and 2.6 (they aren't looked up on the class first but on the instance first like normal members). This makes the following technique still the correct way to mock the with statement.

>>> from mock import Mock
>>> mock = Mock()
>>> mock.__getitem__ = Mock()
>>> mock.__getitem__.return_value = 'bar'
>>> mock['foo']
'bar'
>>> mock.__getitem__.assert_called_with('foo')

mock = Mock()
mock.__enter__ = Mock()
mock.__exit__ = Mock()
mock.__exit__.return_value = False

with mock as m:
    self.assertEqual(m, mock.__enter__.return_value)
mock.__enter__.assert_called_with()
mock.__exit__.assert_called_with(None, None, None)

ConfigObj 4.7.0 (beta): Faster and New Features for Validation

The implementation of the next version of ConfigObj is now complete. All that is remaining is for me to update the documentation and package the release. It will probably be a few days before I can complete the release, which gives you the ideal opportunity for you to try out the new version and check I haven't broken anything.

ConfigObj is a configuration file reader and writer, using ini-style syntax (key = value pairs divided into sections). ConfigObj is easier to use than the standard library ConfigParser, plus ConfigObj has a host of powerful extra features. These features include:

• Nested sections (subsections), to any level

• List values and multi-line values

• String interpolation (substitution)

• Integrated with a powerful validation system

  • including automatic and extensible type checking/conversion
  • repeated sections
  • and allowing default values

• When writing out config files, ConfigObj preserves all comments and the order of members and sections

• Many useful methods and options for working with configuration files (like the 'reload' and 'merge' methods)

• Full Unicode support

The new version, 4.7.0, has had a major performance boost (~25%) thanks to a patch supplied by Christian Heimes. It also has a few bugfixes and some new features for validation. If you use ConfigObj please download the new version and give it a try.

• configobj module in svn repository
• changelog

It's definitely worth trying out this version as it includes dropping support for Python 2.2, a minor grammar change and deprecating passing a dictionary of options to the ConfigObj constructor. If you use validation, particularly with preserve_errors=True then there are also relevant changes you should be aware of.

The full changelog for this version is:

• Minimum supported version of Python is now 2.3
• ~25% performance improvement thanks to Christian Heimes
• String interpolation now works in list value members
• After validation any additional entries not in the configspec are listed in the extra_values section member
• Addition of the get_extra_values function for finding all extra values in a validated ConfigObj instance
• Deprecated the use of the options dictionary in the ConfigObj constructor and added explicit keyword arguments instead. Use **options if you want to initialise a ConfigObj instance from a dictionary
• BUGFIX: Checks that failed validation would not populate 'default_values' and 'restore_default_value' wouldn't work for those entries
• BUGFIX: clear() now clears 'defaults'
• BUGFIX: empty values in list values were accidentally valid syntax. They now raise a ParseError. e.g. "value = 1, , 2"
• BUGFIX: Change to the result of a call to validate when preserve_errors is True. Previously sections where all values failed validation would return False for the section rather than preserving the errors. False will now only be returned for a section if it is missing
• Distribution includes version 1.0.1 of validate.py
• Removed __revision__ and __docformat__

In this version I also gave up on doctest and started using unittest for new tests. There may be ways round the limitations of doctest (how do you specify a single test to run? easy way of adding print debug statements without breaking every test that hits that code-path?), but I just find unittest so much easier to work with.

Thanks to all who reported bugs or provided patches.

• Front page
• All By Date
• All By Category
• Blog Statistics
• RSS Feed
• Return to Top

Technorati...
This work is licensed under a Creative Commons Attribution-Share Alike 2.0 License.

Counter...

• IronPython in Action

  

  IronPython in Action is the first English book on IronPython. It is for Python and .NET programmers interested in making the most of Python on the .NET framework and Mono.

• Search Site

  	Search this Site:
  	www voidspace.org.uk

• Browsing

  Home > Python Index > Techie Blog

• Blogads

  Blogads

• Michael Foord

  • Contact Michael
  • Michael on Twitter
  • My bookmarks
  • Shared articles
  • Photos on Flickr
  • Voidspace Amazon Store
  • Amazon Computers: Todays Deals

• Hosting