Python Programming, news on the Voidspace Python Projects and all things techie.

mock: patching django settings and handling a second call

emoticon:world I still haven't released mock 0.7.0 final. This is waiting for me to find a clear day to work on the documentation. Hopefully I will find time during the Christmas week shutdown at Canonical. (The first time in years that I've had a clear week break for Christmas.)

In the meantime I've fixed a new bug (but no new features!). I've occasionally had complaints that mock.patch doesn't work for patching the django settings object [1].

As this has bitten me at work I decided to work out why.

The django LazySettings object is a proxy object that forwards attribute access. It is built on top of LazyObject.

A good example equivalent, which I use for testing, looks like this:

def get_proxy(obj):
    class Proxy(object):
        def __getattr__(self, name):
            return getattr(obj, name)
        def __setattr__(self, name, value):
            setattr(obj, name, value)
        def __delattr__(self, name):
            delattr(obj, name)
    return Proxy()

If we try to patch a proxy object like this then unpatching (with any version of mock except the latest in the mercurial repository) will fail:

>>> from mock import patch
>>> class X(object):
...     foo = 'bar'
...
>>> proxy = get_proxy(X)
>>> with patch.object(proxy, 'foo', 'baz'):
...     assert proxy.foo == 'baz'
...
>>> X.foo
Traceback (most recent call last):
  ...
AttributeError: type object 'X' has no attribute 'foo'

Instead of the original being reset the attribute is deleted instead! This can make later tests fail for inexplicable reasons, so it is very bad. Ouch.

The reason for this is that the object we are patching doesn't 'own' the attribute we are patching. Normally when this happens it is because the patch is creating an attribute that shadows the real one; for example we are patching an instance to shadow a class attribute. In those cases merely deleting the attribute 'undoes' the shadowing and restores the original.

The fix is relatively simple and is committed in the repository. Now after patch deletes an attribute to undo a patching it checks that the attribute is then available. If it isn't patch restores the original as it would do for a local attribute. One exception is when patch creates the attribute in the first place, in which case deleting it does restore the original situation. That only happens if you pass create=True when you create the patch.

It is still possible to construct proxy objects that this doesn't work for, but solving the general case isn't possible (unless patch can analyse the object it is patching and determine its behaviour). If you don't want the original attribute to be restored after the patch is deleted then you can explicitly pass create=True and this covers the vast majority of possible cases.

As it would be nice to have this fix available before I get around to the final release I may do a 0.7.0 RC.

As with my last couple of blog entries on mock this entry covers two topics. This second one is a funky little pattern for handling mocking functions that need to behave differently on subsequent calls. In my case I needed to test fallback behaviour when posting a notification fails. If the first post fails we need to retry with a different set of parameters. To test the fallback behaviour I want to patch out the code that posts the notification with a mock that raises an exception the first time it is called and returns a valid response the second time. If the second call is made with the correct parameters I know the fallback code works.

The way I did this was with a side_effect function that replaces itself. The first time it is called the side_effect sets a new side_effect that will be used for the second call. It then raises an exception:

def side_effect(*args):
    def second_call(*args):
        return 'response'
    mock.side_effect = second_call
    raise Exception('boom')
>>> mock = Mock(side_effect=side_effect)
>>> mock('first')
Traceback (most recent call last):
  ...
Exception: boom
>>> mock('second')
'response'
>>> mock.assert_called_with('second')

Another perfectly valid way (suggested by my colleague Michael Nelson - aka noodles - after I'd already implemented the solution above) would be to pop return values from a list. If the return value is an exception, raise it instead of returning it:

>>> returns = [Exception('boom'), 'response']
>>> def side_effect(*args):
...   result = returns.pop(0)
...   if isinstance(result, Exception):
...     raise result
...   return result
...
>>> mock = Mock(side_effect=side_effect)
>>> mock('first')
Traceback (most recent call last):
  ...
Exception: boom
>>> mock('second')
'response'
>>> mock.assert_called_with('second')

Which approach you prefer is a matter of taste. The first approach is actually a line shorter but maybe the second approach is more readable.

[1]I've also been bitten by the fact that import settings and the preferred-I-believe from django.conf import settings return different objects. If you are using libraries / apps that do both you may have to patch both. Grrr...

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2010-12-09 10:41:21 | |

Categories: , , Tags: ,


Hosted by Webfaction

Counter...