Python Programming, news on the Voidspace Python Projects and all things techie.

unittest.mock and mock 1.0 alpha 1

emoticon:key One of the results of the Python Language Summit at PyCon 2012 is that mock is now in the Python standard library. In Python 3.3 mock is available as unittest.mock.

For those of you who may not know, mock is a library for testing in Python. It allows you to replace parts of your system under test with mock objects.

The standard distribution of mock that you all know and love will still be available as a backport of the standard library version, but the standard library is now where new mock development will take place. Inevitably this means development will slow down, with new feature releases following the Python release cycle. As the api has stabilised with the 0.8 release this is a good thing...

However, there were some changes I wanted to make (removing obsolete code and adding a few minor features) before moving into the standard library. This is now complete, and there is a shiny new 1.0 alpha 1 release that matches the version in the Python standard library.

You can install mock 1.0 alpha 1 with:

pip install -U mock==dev
easy_install mock

Changes since 0.8 are:

  • mocksignature, along with the mocksignature argument to patch, removed
  • Support for deleting attributes (accessing deleted attributes will raise an AttributeError)
  • Added the mock_open helper function for mocking the builtin open
  • __class__ is assignable, so a mock can pass an isinstance check without requiring a spec
  • Addition of PropertyMock, for mocking properties
  • MagicMocks made unorderable by default (in Python 3). The comparison methods (other than equality and inequality) now return NotImplemented
  • Propagate traceback info to support subclassing of _patch by other libraries
  • BUGFIX: passing multiple spec arguments to patchers (spec , spec_set and autospec) had unpredictable results, now it is an error
  • BUGFIX: using spec=True and create=True as arguments to patchers could result in using DEFAULT as the spec. Now it is an error instead
  • BUGFIX: using spec or autospec arguments to patchers, along with spec_set=True did not work correctly
  • BUGFIX: using an object that evaluates to False as a spec could be ignored
  • BUGFIX: a list as the spec argument to a patcher would always result in a non-callable mock. Now if __call__ is in the spec the mock is callable

Documentation for 1.0a1 can be found at mock.readthedocs.org.

This includes one backwards incompatible change - the removal of mocksignature in all its guises. Normally I would only remove features with a deprecation period, but I wanted it gone before moving mock in the standard library. mocksignature is completely replaced by the create_autospec function and the autospec argument to patch.

There are a whole series of bugfixes around the spec arguments to the patchers. I discovered these whilst taking a look at improving a couple of error messages. That was a rabbit hole that absorbed an entire Sunday afternoon!

There are still two issues I'd like to fix for Python 3.3, but no guarantees:

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2012-03-28 18:15:44 | |

Categories: , Tags: ,


Ergonomics: Kinesis Freestyle Keyboard and Evoluent Vertical Mouse

emoticon:ghostradio I've been using computers for a long time, and for most of that time I've been using them for the whole of the working day and often the rest of the day too. A few years ago I started getting pains in my wrists (classic programmer's RSI) and began using wrist rests and an ergonomic keyboard.

Without these accessories my wrists begin to hurt within about ten minutes of typing. With good ergonomic gear I can spend all day every day slaving over a warm keyboard and not worry about it. Because they fix the bug in my DNA I've never taken the problem to a doctor nor had any professional diagnosis. What follows here is purely based on my experiences. If you have RSI, or are starting to get pains when you type, seeking professional advice is a much better idea than paying attention to anything I might write...

Ergonomic keyboards aren't just about comfort or combatting RSI. One of the ways they reduce stress on the hands and wrists is by minimising movements needed to reach the keys. This means you can type faster. They often achieve this with unusual keyboard shapes, which makes touch typing essential. So as an added side effect your touch typing will have to improve.

My first ergonomic keyboard was a Microsoft Natural 4000. I've always been happy with it as a keyboard, and compared to many ergonomic keyboards it's very cheap. It's main drawback is that with the palm rest in place it's pretty big.

One PyCon Chris Perkins introduced me to the Kinesis Advantage keyboard. You can see from this photo that it's a very odd looking keyboard.

The Kinesis Advantage Keyboard

The Kinesis advantage is basically a plastic block with two wells for the keys, along with palm rests just below the wells. With your palms on the rests your hands are in the wells, and you can reach every key, including arrow keys, without moving your hands. The downside is that you can't see the keys and you have to touch type. Once you get used to the layout you can type like the wind. I love this keyboard, I consider it the Rolls Royce of keyboards. I can type pretty much all day all week without a hint of wrist ache. The build quality of the Kinesis is "solid", but not pretty.

The UK distributor for the Kinsesis Advantage is Osmond Ergonomics. After purchasing the, very expensive, Advantage keyboard I followed Osmond on twitter and had several exchanges with Guy Osmond who runs the twitter account. He emailed me with a suggestion that I review some of their other ergonomic products. They sent me these products free of charge, but with my stipulation that I have complete editorial freedom to speak my mind in the reviews.

Note

There's more to ergonomics than just RSI. Osmond have a website offering Posture guidance and exercises.

Osmond Ergonomics are offering a 10% for readers of this blog. Use this code on checkout: TFM022812.

Ergonomic computer equipment is of a great deal of interest to programmers. Particularly Python programmers in my circle of twitter friends / mailing lists where many of the programmers spend their free time hacking as well as their paid hours.

The two products I've been trying are the Evoluent vertical mouse and the Kinesis Freestyle split keyboard.

I've always wanted to try a split keyboard. When I travel for conferences and sprints I take a mobile workstation with me, including the Microsoft Natural Keyboard. So my question for the Freestyle was, is it good enough to replace the Advantage as my main keyboard? If not is it good enough to replace the Natural on my travels?

The unboxing:

Unboxing the Kinesis Freestyle Keyboard. From its box.

The version of the Freestyle I tried had the "VIP" stand attached. You can also use it flat or in a vertical configuration.

So my initial impressions are good. The build quality is a great improvement on my Advantage. Maybe after two years new Advantage keyboards are also better. The version I'm trying out is the PC layout rather than the Mac one, but after switching the Alt and Super keys around (in the Mac keyboard preferences) it works fine. [1]

The Kinesis Freestyle Keyboard on the VIP stand

I like the angle of the keyboard with the stand, but the raised height means I still need to use a wrist rest. Maybe I should try it flat. I've been using the Freestyle as my main keyboard now for two weeks. Because the layout is different from the Advantage (both qwerty - and both support dvorak of course - but the numbers etc are in different place) it took me a while to get used to it. Unlike the Advantage some of the keys, particularly the cursor keys, need hand movement to use. The Freestyle has a space bar on both sides of the split, which is a nice touch as you use alternate thumbs for space when touch typing. Beyond this the Freestyle is a nice and straightforward keyboard with decent key action.

After a weeks typing I do notice mild wrist ache, and I find having to move my hands to use the cursor keys more disruptive than the Advantage. The Freestyle will definitely displace the Microsoft Natural for my travels (the Natural is a huge keyboard and with the split the Freestyle is much more convenient to pack), but for day-to-day use I'll stick with the Advantage. For anyone with mild RSI, or a penchant for decent keyboards, I can heartily recommend the Kinesis Freestyle.

Some other notes about keyboards. Both the Advantage and Freestyle are wired keyboards. I would love to have either of them as wireless keyboards. Neither have number pads (well, with both you can switch part of the keyboard to be a number pad but this is fiddly). Many geeks don't like number pads and see them as unnecessary. Having worked in a builders merchant for a few years I'm pretty speedy with the number pad, much faster than with standard number keys, and so I have a separate USB number pad.

The second device I tried is the Evoluent Vertical Wireless Mouse. In my try-to-stop-my-wrist-hurting-when-I-use-the-computer adventures I first switched to a mouse mat with a wrist wrest and finally a Kensington Trackball (Slimblade). The Evoluent is an optical mouse, available wired and wireless and with left hand and right hand models.

The Evoluent Vertical Mouse alongside a Kensington Trackball

The trackball is great, and again solves all my wrist pain associated with mouse use. I do need to use a wrist rest with it so that my wrist can be fully rested whilst using the trackball, but pointer movement requires no wrist movements and finger movements are very small. I was sceptical that any mouse could be as good, but I don't take the trackball with me when I travel and have been using an apple magic mouse. The combination of multi-touch and mouse is innovative, and very well done, but I do get some pain from the magic mouse after a week long sprint. What I was looking for in the Evoluent was a mouse that could displace the magic mouse when I travel, and I was curious about whether a "vertical mouse" offered the ergonomic benefits it claimed.

First of all, because you hold the mouse vertically, it is physically bigger than I anticipated. The one I tried isn't the Mac model and has its own wireless dongle. Apparently a Mac version, using bluetooth, will be available soon. There is no Mac driver available for the Evoluent Wireless, and they warn that the multiple buttons it provides may not be useable out of the box. I use the Steermouse driver anyway, as I've tried several different mice over the years. This recognised the Evoluent and it worked with zero configuration. (I use Steermouse to reprogram the middle mouse button [2].)

To my surprise I love the Evoluent mouse, to the extent that I'm using it instead of the trackball. Holding the mouse vertically means my arm and wrist are fully rested whilst holding the mouse and there is no need for any kind of external rest. Pressing the middle button with my middle finger, instead of moving my index finger took a little bit to get used to, but there is button by the thumb grip which I reprogrammed to act as a middle button too. The Evoluent is a very well built product, not too heavy yet feeling robust and not "cheap" (which it isn't).

Because of its size, and that it would take up a precious usb port on my laptop, I'm not yet sure if I'll take the Evoluent on my travels.

In the first week of using it I was finding the pointer would get stuck every half hour or so and take a bit of "wiggling" to un-stick. This was very annoying and spoiled my enjoyment of using the mouse. At first I thought it was because of using an optical mouse on a shiny desk, so I switched to a large mouse mat. This didn't help. Eventually I twigged that it was because the wireless dongle was plugged into the back of my computer, about three foot from the mouse and under the desk. Putting the wireless dongle into my USB hub (about a foot from the mouse) completely solved the problem. The mouse works fine on the desk, but I've kept the mouse mat in place anyway.

In conclusion both the Freestyle keyboard and the Evoluent mice are great devices and whether or not you have RSI you're likely to enjoy using them. They both work fine with Windows and the Mac, with the caveats described above. I didn't investigate Linux compatibility as although I do all my day to day development in Ubuntu, it's inside a VM. For my day to day use I'm sticking with the Kinesis Advantage, but I am switching to the Evoluent as my everyday mouse.

These devices were supplied free of charge by Osmond ergonomics for me to review, however this review is my own opinion. If you have any questions, or suffer from RSI, Osmond would be very happy to talk to you. If you do order from them, don't forget the voucher for a 10% discount: TFM022812.


[1]Which leads me to a grouse with the Advantage, pretty much my only grouse with the Advantage. The Advantage is highly programmable, which I generally don't use, but because you program it on-board it doesn't work well with the Mac keyboard preferences. Specifically switching off the caps-lock doesn't work and I have to remap the caps-lock key to something innocuous.
[2]In conjunction with Deja menu so I can access the main menu bar on any monitor with a middle mouse click.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2012-03-05 10:16:23 | |

Categories: , Tags: , ,


Tests that fail one day every four years

emoticon:computer Some code looks harmless but has hidden bugs lurking in its nether regions. Code that handles dates is notorious for this, and this being February 29th (the coders' halloween) it's time for the bugs to come crawling out of the woodwork.

Some of our tests were constructing expected dates, one year from today, with the following code:

from datetime import datetime

now = datetime.utcnow()
then = datetime(now.year + 1, now.month, now.day)

Of course if you run this code today, then it tries to construct a datetime for February 29th 2013, which fails because that date doesn't exist.

When I posted this on twitter a few people suggested that instead we should have used timedelta(days=365) instead. Again, this works most of the time - but if you want a date exactly one year from now it will fail in leap years when used before February 29th:

>>> from datetime import datetime, timedelta
>>> datetime(2012, 2, 27) + timedelta(days=365)
datetime.datetime(2013, 2, 26, 0, 0)

The correct fix is to use the wonderful dateutil module, in particular the dateutil.relativedelta.relativedelta:

>>> from datetime import datetime
>>> from dateutil.relativedelta import relativedelta
>>> datetime.utcnow() + relativedelta(years=1)
datetime.datetime(2013, 2, 28, 15, 20, 21, 546755)
>>> datetime(2012, 2, 27) + relativedelta(years=1)
datetime.datetime(2013, 2, 27, 0, 0)

And as another hint, always use datetime.utcnow() instead of datetime.now() to avoid horrible timezone nightmares (exactly which timezone are your servers in?).

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2012-02-29 15:24:39 | |

Categories: , Tags: ,


mock 0.8 released

emoticon:beaker After more than six months development work mock 0.8 has been released. 0.8 is a big release with many new features, general improvements and bugfixes.

You can download mock 0.8.0 final from the PyPI page or install it with:

pip install -U mock

mock is a library for testing in Python. It allows you to replace parts of your system under test with mock objects.

The only changes in mock 0.8.0 final since 0.8rc2 are:

  • Improved repr of sentinel objects
  • ANY can be used for comparisons against call objects
  • The return value of MagicMock.__iter__ can be set to any iterable and isn't required to be an iterator

The full changelog since 0.7.2 is pretty large. Hopefully I'll do a series of blog posts explaining the major new features and how to use them. Several common patterns of testing with mock have become simpler in 0.8.

Several of the changes, particularly the improved reprs and error messages, come for free without you having to know anything about the other new features. For documentation on the new features, browse the links in the changelog.

Here's a brief example of the improved reprs / failure messages you get with mock 0.8:

>>> from mock import MagicMock
>>> m = MagicMock(name='foo')
>>> m.method(1, 2, 3).attribute['foo']
<MagicMock name='foo.method().attribute.__getitem__()' id='4300665616'>
>>> m.method.call_args
call(1, 2, 3)
>>> m.method.assert_called_with('some args')
Traceback (most recent call last):
  ...
AssertionError: Expected call: method('some args')
Actual call: method(1, 2, 3)

One of the best new features, for making assertions about several calls at once, is mock_calls in conjunction with the call object:

>>> from mock import MagicMock, call
>>> m = MagicMock(name='foo')
>>> config = {'method.return_value.chained.return_value.nested.return_value': 3}
>>> m.configure_mock(**config)
>>> m.method('arg').chained().nested('call')
3
>>> m.mock_calls
[call.method('arg'),
 call.method().chained(),
 call.method().chained().nested('call')]
>>> m.mock_calls == [call.method('arg'),
...  call.method().chained(),
...  call.method().chained().nested('call')]
True
>>> expected_calls = call.method('arg').chained().nested('call').call_list()
>>> m.mock_calls == expected_calls
True

If you're still using mock 0.7, and can't upgrade all your test code yet, the 0.7 documentation is available online here.

The full List of changes since 0.7:

mock 0.8.0 is the last version that will support Python 2.4.

  • Addition of mock_calls list for all calls (including magic methods and chained calls)
  • patch and patch.object now create a MagicMock instead of a Mock by default
  • The patchers (patch, patch.object and patch.dict), plus Mock and MagicMock, take arbitrary keyword arguments for configuration
  • New mock method configure_mock for setting attributes and return values / side effects on the mock and its attributes
  • New mock assert methods assert_any_call and assert_has_calls
  • Implemented auto-speccing (recursive, lazy speccing of mocks with mocked signatures for functions/methods), as the autospec argument to patch
  • Added the create_autospec function for manually creating 'auto-specced' mocks
  • patch.multiple for doing multiple patches in a single call, using keyword arguments
  • Setting side_effect to an iterable will cause calls to the mock to return the next value from the iterable
  • New new_callable argument to patch and patch.object allowing you to pass in a class or callable object (instead of MagicMock) that will be called to replace the object being patched
  • Addition of NonCallableMock and NonCallableMagicMock, mocks without a __call__ method
  • Addition of mock_add_spec method for adding (or changing) a spec on an existing mock
  • Protocol methods on MagicMock are magic mocks, and are created lazily on first lookup. This means the result of calling a protocol method is a MagicMock instead of a Mock as it was previously
  • Addition of attach_mock method
  • Added ANY for ignoring arguments in assert_called_with calls
  • Addition of call helper object
  • Improved repr for mocks
  • Improved repr for call_args and entries in call_args_list, method_calls and mock_calls
  • Improved repr for sentinel objects
  • patch lookup is done at use time not at decoration time
  • In Python 2.6 or more recent, dir on a mock will report all the dynamically created attributes (or the full list of attributes if there is a spec) as well as all the mock methods and attributes.
  • Module level FILTER_DIR added to control whether dir(mock) filters private attributes. True by default.
  • patch.TEST_PREFIX for controlling how patchers recognise test methods when used to decorate a class
  • Support for using Java exceptions as a side_effect on Jython
  • Mock call lists (call_args_list, method_calls & mock_calls) are now custom list objects that allow membership tests for "sub lists" and have a nicer representation if you str or print them
  • Mocks attached as attributes or return values to other mocks have calls recorded in method_calls and mock_calls of the parent (unless a name is already set on the child)
  • Improved failure messages for assert_called_with and assert_called_once_with
  • The return value of the MagicMock.__iter__ method can be set to any iterable and isn't required to be an iterator
  • Added the Mock API (assert_called_with etc) to functions created by mocksignature
  • Tuples as well as lists can be used to specify allowed methods for spec & spec_set arguments
  • Calling stop on an unstarted patcher fails with a more meaningful error message
  • Renamed the internal classes Sentinel and SentinelObject to prevent abuse
  • BUGFIX: an error creating a patch, with nested patch decorators, won't leave patches in place
  • BUGFIX: __truediv__ and __rtruediv__ not available as magic methods on mocks in Python 3
  • BUGFIX: assert_called_with / assert_called_once_with can be used with self as a keyword argument
  • BUGFIX: when patching a class with an explicit spec / spec_set (not a boolean) it applies "spec inheritance" to the return value of the created mock (the "instance")
  • BUGFIX: remove the __unittest marker causing traceback truncation
  • Removal of deprecated patch_object
  • Private attributes _name, _methods, '_children', _wraps and _parent (etc) renamed to reduce likelihood of clash with user attributes.
  • Added license file to the distribution

Thanks to all those who suggested features, provided patches and helped test the mock 0.8 release. There are still plenty of ways mock can continue to improve, you can browse the issues list to get an idea of some of the outstanding feature requests and suggestions.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2012-02-16 12:56:14 | |

Categories: , Tags: , ,


Callable object with state using generators

emoticon:globesearch It's often convenient to create callable objects that maintain some kind of state. In Python we can do this with objects that implement the __call__ method and store the state as instance attributes. Here's the canonical example with a counter:

>>> class Counter(object):
...     def __init__(self, start):
...         self.value = start
...     def __call__(self):
...         value = self.value
...         self.value += 1
...         return value
...
>>> counter = Counter(0)
>>> counter()
0
>>> counter()
1
>>> counter()
2

Generators can be seen as objects that implement the iteration protocol, but maintain state within the generator function instead of as instance attributes. This makes them much simpler to write, and much easier to read, than manually implementing the iteration protocol.

This example iterator never terminates, so we obtain values by manually calling next:

>>> class Iter(object):
...     def __init__(self, start):
...         self.value = start
...     def __iter__(self):
...         return self
...     def next(self):
...         value = self.value
...         self.value += 1
...         return value
...
>>> counter = Iter(0)
>>> next(counter)
0
>>> next(counter)
1
>>> next(counter)
2

The same iterator implemented as a generator is much simpler and the state is stored as local variables in the generator:

>>> def generator(start):
...     value = start
...     while True:
...         yield value
...         value += 1
...
>>> gen = generator(0)
>>> next(gen)
0
>>> next(gen)
1
>>> next(gen)
2

In recent versions of Python generators were enhanced with a send method to enable them to act like coroutines.

>>> def echo():
...     result = None
...     while True:
...         result = (yield result)
...
>>> f = echo()
>>> next(f)  # initialise generator
>>> f.send('fish')
'fish'
>>> f.send('eggs')
'eggs'
>>> f.send('ham')
'ham'

(Note that we can't send to an unstarted generator - hence the first call to next to initialise the generator.)

We can use the send method as a way of providing a callable object with state. I first saw this trick in this recipe for a highly optimized lru cache by Raymond Hettinger. The callable object is the send method itself, and as with any generator the state is stored as local variables.

Here's our counter as a generator:

>>> def counter(start):
...     yield None
...     value = start
...     while True:
...         ignored = yield value
...         value += 1
...
>>> gen = counter(0)
>>> next(gen)
>>> f = gen.send
>>> f(None)
0
>>> f(None)
1
>>> f(None)
2
>>> f(None)
3

Some observations. Firstly send takes one argument and one argument only. In this example we're ignoring the value sent into the generator, but send must be called with one argument and can't be called with more. So it's mostly useful for callable objects that take a single argument...

Secondly, this performs very well. Function calls are expensive (relatively) in Python because each invocation creates a new frame object (or reuses a zombie frame from the pool - but I digress) for storing the local variables etc. Generators are implemented with a "trick" that keeps the frame object alive, so that the next step of the generator can simply continue execution after the last yield. So our callable object implemented as a generator doesn't have the overhead of a normal function call...

The main advantage this approach has is that it's more readable than the version with __call__. To make it more pleasant to use, we can wrap creating our counter in a convenience function:

>>> def get_counter(start):
...     c = counter(start)
...     next(c)
...     return c.send
...
>>> c = get_counter(0)
>>> c(None)
0
>>> c(None)
1
>>> c(None)
2

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2012-01-22 15:05:46 | |

Categories: , Tags: ,


Simple mocking of open as a context manager

emoticon:scanner Using open as a context manager is a great way to ensure your file handles are closed properly and is becoming common:

with open('/some/path', 'w') as f:
    f.write('something')

The issue is that even if you mock out the call to open it is the returned object that is used as a context manager (and has __enter__ and __exit__ called).

Using MagicMock from the mock library, we can mock out context managers very simply. However, mocking open is fiddly enough that a helper function is useful. Here mock_open creates and configures a MagicMock that behaves as a file context manager.

from mock import inPy3k, MagicMock

if inPy3k:
    file_spec = ['_CHUNK_SIZE', '__enter__', '__eq__', '__exit__',
        '__format__', '__ge__', '__gt__', '__hash__', '__iter__', '__le__',
        '__lt__', '__ne__', '__next__', '__repr__', '__str__',
        '_checkClosed', '_checkReadable', '_checkSeekable',
        '_checkWritable', 'buffer', 'close', 'closed', 'detach',
        'encoding', 'errors', 'fileno', 'flush', 'isatty',
        'line_buffering', 'mode', 'name',
        'newlines', 'peek', 'raw', 'read', 'read1', 'readable',
        'readinto', 'readline', 'readlines', 'seek', 'seekable', 'tell',
        'truncate', 'writable', 'write', 'writelines']
else:
    file_spec = file

def mock_open(mock=None, data=None):
    if mock is None:
        mock = MagicMock(spec=file_spec)

    handle = MagicMock(spec=file_spec)
    handle.write.return_value = None
    if data is None:
        handle.__enter__.return_value = handle
    else:
        handle.__enter__.return_value = data
    mock.return_value = handle
    return mock
>>> m = mock_open()
>>> with patch('__main__.open', m, create=True):
...     with open('foo', 'w') as h:
...         h.write('some stuff')
...
>>> m.assert_called_once_with('foo', 'w')
>>> m.mock_calls
[call('foo', 'w'),
 call().__enter__(),
 call().write('some stuff'),
 call().__exit__(None, None, None)]
>>> handle = m()
>>> handle.write.assert_called_once_with('some stuff')

And for reading files, using a StringIO to represent the file handle:

>>> from StringIO import StringIO
>>> m = mock_open(data=StringIO('foo bar baz'))
>>> with patch('__main__.open', m, create=True):
...     with open('foo') as h:
...         result = h.read()
...
>>> m.assert_called_once_with('foo')
>>> assert result == 'foo bar baz'

Note that the StringIO will only be used for the data if open is used as a context manager. If you just configure and use mocks they will work whichever way open is used.

This helper function will be built into mock 0.9.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2012-01-13 12:18:35 | |

Categories: , , Tags: , ,


Mocks with some attributes not present

emoticon:movies Mock objects, from the mock library, create attributes on demand. This allows them to pretend to be objects of any type.

What mock isn't so good at is pretending not to have attributes. You may want a mock object to return False to a hasattr call, or raise an AttributeError when an attribute is fetched. You can do this by providing an object as a spec for a mock, but that isn't always convenient.

Below is a subclass of Mock that allows you to "block" attributes by deleting them. Once deleted, accessing an attribute will raise an AttributeError.

from mock import Mock

deleted = object()
missing = object()

class DeletingMock(Mock):
    def __delattr__(self, attr):
        if attr in self.__dict__:
            return super(DeletingMock, self).__delattr__(attr)
        obj = self._mock_children.get(attr, missing)
        if obj is deleted:
            raise AttributeError(attr)
        if obj is not missing:
            del self._mock_children[attr]
        self._mock_children[attr] = deleted

    def __getattr__(self, attr):
        result = super(DeletingMock, self).__getattr__(attr)
        if result is deleted:
            raise AttributeError(attr)
        return result
>>> mock = DeletingMock()
>>> hasattr(mock, 'm')
True
>>> del mock.m
>>> hasattr(mock, 'm')
False
>>> del mock.f
>>> mock.f
Traceback (most recent call last):
  ...
AttributeError: f

This functionality will probably be built into 0.9, or whichever version of mock comes after 0.8...

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2012-01-12 12:33:15 | |

Categories: , , Tags: , ,


mock 0.8rc2: new release and development docs

emoticon:baldguy I've pushed out a new release of mock. This fixes an inconsistency in the create_autospec api I discovered whilst working on the docs (yes I've really been working on the docs), and a fix for a bug with using ANY. (Thanks to Tom Davis for reporting this.)

You can download mock 0.8rc2 or install it with:

pip install -U mock==dev

mock is a library for testing in Python. It allows you to replace parts of your system under test with mock objects. The latest stable release is 0.7.2, which you can download from pypi.

There are two changes in 0.8rc2:

  • Removed the configure keyword argument to create_autospec and allow arbitrary keyword arguments (for the Mock constructor) instead
  • Fixed ANY equality with some types in assert_called_with calls

Unfortunately, if you were using the configure argument to create_autospec then you'll have to change code when you update to 0.8rc2, sorry. The good news is that there's now some documentation for the new features - including for create_autospec.

I've now switched mock to using one of the standard Sphinx themes, and the development documentation (updated automatically on every commit) is available at mock.readthedocs.org.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2012-01-11 02:13:32 | |

Categories: , Tags: , ,


Python on Google Plus

emoticon:film As you may (or perhaps not) have noticed, I've been blogging a lot less in the last year. A new job with Canonical (although I've been there over a year now) and an eight month old daughter all make blogging harder. For Python news I've been posting more on Google+:

I've made some interesting posts on PyPy, Python 3, interesting new libraries and other snippets of news. I particularly try and post about new PEPs and big changes to Python.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2012-01-03 11:41:02 | |

Categories: , ,


Sphinx doctests and the execution namespace

emoticon:globesearch I've finally started work on the documentation for mock 0.8 release, and much of it involves converting the write-ups I did in the blog entries.

The mock documentation is built with the excellent Sphinx (of course!) and as many as possible of the examples in the documentation are doctests, to ensure that the examples are still up to date for new releases.

doctests mimic executing your examples at the interactive interpreter, but they aren't exactly the same. One big difference is that the execution context for a doctest is not a real module, but a dictionary. This is particularly important for mock examples, because the following code will work at the interactive interpreter but not in a doctest:

>>> from mock import patch
>>> class Foo(object):
...      pass
...
>>> with patch('__main__.Foo') as mock_foo:
...   assert Foo is mock_foo
...

The name (__name__) of the doctest execution namespace is __builtin__, but this is a lie. The namespace is a dictionary, internal to the DocTest. Whilst executing doctests under Sphinx, the real __main__ module is the sphinx-build script.

To get the example code above working I either need to rewrite it (and make it less readable - probably by shoving the class object I'm patching out into a module), or I need to somehow make the current execution context into __main__.

Fortunately the Sphinx doctest extension provides the doctest_global_setup config option. This allows me to put a string into my conf.py, which will be executed before the doctests of every page in my documentation (doctests from each page share an execution context).

I solved the problem by creating a proxy object that delegates attribute access to the current globals() dictionary. I shove this into sys.modules as the __main__ module (remembering to store a reference to the real __main__ so that it doesn't get garbage collected). When patch accesses or changes an attribute on __main__ it actually uses the current execution context.

Here's the code from conf.py:

doctest_global_setup = """
import sys
import __main__

# keep a reference to __main__
__main = __main__

class ProxyModule(object):
    def __getattr__(self, name):
        return globals()[name]
    def __setattr__(self, name, value):
        globals()[name] = value
    def __delattr__(self, name):
        del globals()[name]

sys.modules['__main__'] = ProxyModule()
"""

doctest_global_cleanup = """
sys.modules['__main__'] = __main
"""

The corresponding doctest_global_cleanup option restores the real __main__ when the test completes.

Note

In the comments Nick Coghlan suggests a simplification for the ProxyModule:

class ProxyModule(object):
    def __init__(self):
        self.__dict__ = globals()

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2012-01-01 00:28:03 | |

Categories: , Tags: , ,


mock 0.8 release candidate 1 and handling mutable arguments

emoticon:cross I've released mock 0.8 release candidate 1. You can download it it or install it with:

pip install -U mock==dev

mock is a library for testing in Python. It allows you to replace parts of your system under test with mock objects. The latest stable release is 0.7.2, which you can download from pypi.

The main improvement in 0.8 rc1 is performance improvements, particularly in constructing MagiMock. Instantiating MagicMock is substantially faster than it was in 0.7 already, however patch creates a MagicMock by default now (in 0.7 patch creates a Mock by default). Instantiating MagicMock in 0.8 is still slower than instantiating a Mock in 0.7, but the difference is much less than it used to be.

There are also a couple of bugfixes. The full changelog for 0.8 rc1 is:

  • create_autospec on the return value of a mock class will use __call__ for the signature rather than __init__
  • Performance improvement instantiating Mock and MagicMock
  • Mocks used as magic methods have the same type as their parent instead of being hardcoded to MagicMock

The additional changes in 0.8 are explained in these blog entries:

Special thanks to Julian Berman for his help with diagnosing and improving performance in this release. I have started work on the doc changes and I'm hoping that the 0.8 final release won't be too far off.

Mock, and its now multifarious variants, are designed to "remember" arguments they are called with for making assertions about how they have been used. There are two ways it could do this: it could either keep references to the arguments or it could copy them.

Mock chooses to keep a reference to the arguments rather than copying them, mainly because this was easier to implement and met all my use cases at the time. As a consequence, if a mock is called with mutable arguments (like a set or a dictionary) that are changed (mutated) in between the call and the assertion then you have to make the assertion against the changed value rather than the value as it was at the call time. Here's an example of what I mean:

>>> from mock import MagicMock
>>> arg = set()
>>> m = MagicMock(return_value=None)
>>> m(arg)
>>> arg.add(1)
>>> m.assert_called_with(set())
Traceback (most recent call last):
  ...
AssertionError: Expected call: mock(set([]))
Actual call: mock(set([1]))

Even though m was called with an empty set, an assertion with an empty set fails.

The alternative would be to copy all arguments for every call (using copy.deepcopy). Unfortunately deepcopy is slow and fragile, and although it would solve this problem with mutable arguments it would break assertions that rely on object identity.

The mock documentation has an example showing two ways to solve this problem. There's a third way which is possibly more elegant, a subclass of MagicMock that deep-copies all arguments:

>>> from copy import deepcopy
>>> class CopyingMock(MagicMock):
...     def __call__(self, *args, **kwargs):
...         args = deepcopy(args)
...         kwargs = deepcopy(kwargs)
...         return super(CopyingMock, self).__call__(*args, **kwargs)
...
>>> c = CopyingMock(return_value=None)
>>> arg = set()
>>> c(arg)
>>> arg.add(1)
>>> c.assert_called_with(set())
>>> c.assert_called_with(arg)
Traceback (most recent call last):
  ...
AssertionError: Expected call: mock(set([1]))
Actual call: mock(set([]))
>>> c.foo
<CopyingMock name='mock.foo' id='...'>

When you subclass Mock or MagicMock all dynamically created attributes, and the return_value will use your subclass automatically. That means all children of a CopyingMock will also have the type CopyingMock.

This goes nicely with the new-in-0.8 new_callable argument to patch that allows you to specify the mock class to use:

with patch('some.object', new_callable=CopyMock) as mock_thing:
        # do things...

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2011-12-29 13:04:47 | |

Categories: , Tags: , , , , ,


Hosted by Webfaction

Counter...