Dynamic Languages MVP, Blog Template, Conferences, Concurrency and Other Stuff

Last week I received an email from Microsoft:

Congratulations! We are pleased to present you with the 2009 Microsoft MVP Award! This award is given to exceptional technical community leaders who actively share their high quality, real world expertise with others.

MVP stands for 'Most Valued Professional' and is an award given to community contributors. I'm the first MVP for the Microsoft dynamic languages team - although there isn't a dynamic languages division for MVPs so technically I'm a Visual C# (*) MVP. That asterisk is very important!

It wasn't all good news last week though. Menno Smits has been a colleague with us at Resolver Systems for almost a year, but he has just passed away. Well, he received an 'offer too good to refuse' and will now be working with BATS Trading in London. They are a trading platform largely implemented in Python and have been so successful in the states that they have become a stock exchange in their own right. Now they are setting up shop in Europe. Congratulations to Menno and we wish him all the best for the future.

If you're anything more than an occasional visitor to this blog you will probably have noticed the change in template. Largely on the insistence of Christian Muirhead (my colleague and co-author of IronPython in Action) I've chopped out almost everything above the entries - which means less to scroll down through and less Javascript should mean faster loading pages. I'm still intending to do (or at least get Justin to do) a full site redesign once 'the book is finished'. This change will have to do in the mean time.

PyCon UK is done so it's time to look at conferences for next year. I've submitted talks for PyCon 2009 (in Chicago) and ACCU 2009 (in Oxford).

ACCU is a UK community conference (I went for the first time last year and it was great fun). It has talks on a wide variety of topics - last year there were very few Python talks but several on Haskell and Erlang. In the past there have been many more Python talks, but last year a significant proportion of those attending were .NET or C++ developers.

I've proposed two talks (I'm not sure which they will prefer):

• Creating Rich Internet Applications with IronPython & Silverlight 2
• Embedding IronPython and the Dynamic Language Runtime in .NET Applications

I've put forward a talk and a tutorial proposal for PyCon US. My talk was submission number 1 (!) and is on Functional Testing of Desktop Applications. It's a relatively niche subject (not much focus in the Python community on creating desktop applications) - but still an important one, so it will be interesting to see if it is accepted.

The tutorial is on Developing with IronPython. It is based on the tutorial that Menno, Christian and I gave at PyCon UK - except this time it is Jonathan Hartley and I who will be giving it. We had a great time giving the tutorial in Birmingham and learned a great deal doing it. We had seventeen people attending (out of a total of eighty attending the tutorials - so nearly a quarter which isn't bad), and I think that the attendees enjoyed it. We should have got people using IronPython earlier in the tutorial, and we had too much practical stuff - meaning we spent too long in the user interface part of the tutorial. I've been revising the handout notes (I'll post them up here sometime as they are a great introduction to IronPython) based on what we learned.

Interesting advice for new programmers from Anders Hejlsberg (the architect of the C# programming language) in an interview with Computer World:

Go look at dynamic languages and meta-programming: those are really interesting concepts. Once you get an understanding of these different kinds of programming and the philosophies that underlie them, you can get a much more coherent picture of what's going on and the different styles of programming that might be more appropriate for you with what you're doing right now.

Anyone programming today should check out functional programming and meta-programming as they are very important trends going forward.

Before his passing away Menno ported part of his website to rest2web. He also posted a new article on his experiences installing Linux on his shiny new Sony Vaio VGN-BZ11XN notebook. rest2web is a tool for maintaining websites (static HTML) in reStructured Text. It is particularly good for programmers as the templating system is straight Python without requiring you to learn a custom templating language.

I use rest2web to maintain pretty much all the websites I run [1], but haven't done any work on rest2web itself beyond maintenance for the last two years. It simply does everything I need it to. Despite this new sites built on rest2web pop-up regularly. Another recent one is by Andrew Straw. I can always tell a rest2web site, because even with a custom template most people leave in the Page last modified ... timestamp from my default template. This uses the <% modtime %> templating variable. Unfortunately it doesn't play well with a Subversion bug if you keep your website sources under source code control. Andrew has overcome this bug by using the SVN commit time instead and posted notes about how he did it under About this Website.

Final piece of news; both Ted Leung and Mark Shuttleworth talked about the future of Python in their keynote speeches at PyCon UK. They both noted that concurrency was becoming more important and is one of the areas where CPython is lacking because of its poor support for threads. Neither IronPython nor Jython have a Global Interpreter Lock (the GIL), so these are both platforms where threads can be used for concurrency with Python.

Michael Sparks is one of the organizers of PyCon UK, and also the author of Kamaelia, a generator based concurrency library for Python. Kamaelia is capable enough to stream video and audio, but last time Michael tried to use it with IronPython a few bugs (in IronPython) prevented it from working. A lot has changed since then, and with the latest version of IronPython 2 most of Kamaelia 'just works'.

Although Kamaelia presents a very simple API for concurrency oriented programming (usually no need to explicitly work with threads or locking), it does use threads in several key parts under the hood. This means it hits limitations in CPython (Michael Sparks' words not mine), and IronPython doesn't suffer from the same restrictions. So far Michael is impressed with IronPython...

[1]	This one, IronPython in Action, The Other Delia and Resolver Hacks. Exceptions are The IronPython Cookbook which is a MediaWiki wiki and IronPython-URLs Blog which is a blogspot blog (originally started by Mark Rees).

---

Dependency Injection and Mock(ing) for Testing

This blog entry is about changes to Mock - A Lightweight Mocking Framework for Testing, one of the projects (and thankfully small) that I will be working on once 'the book' is finally out of the way.

Suppose you have the following code, how do you test some_method:

Well, we could just call some_method, but what if OtherClass is expensive to construct (or establishes a database connection) and we'd really rather not do that in a unittest?

We can't straightforwardly mock, because the construction of the class is internal to the method. One technique is to use dependency injection, where the dependency class is passed in. We can even make it the default value of a keyword argument:

Now it is trivial to test some_method. We can pass in a mock object as the 'Dependency' and test that the body of the method acts on it as it should. The problem I have with this approach is that we have now added code that sets up the default dependency (the default value of the keyword argument), effectively another layer, that we should test as well. (If we only with an explicitly passed in dependency then we test under different conditions to how the code is called in production.)

This approach has other advantages however. The dependency on OtherClass is no longer 'hardwired' and we could use another class or callable at runtime if we wanted to change the behaviour. Of course this could be catering to a future and purely imagined usecase, and YAGNI applies. There is a lot more to dependency injection than this simple example, and it can be an extremely useful technique, if you need it.

At Resolver Systems we take an alternative approach to testing this kind of code, generally mocking instead of using dependency injection. When I discussed with one of my fellow developers whether we should use more dependency injection to reduce the amount of mocking needed in our tests, his response surprised me:

As far as I can tell, dependency injection just screws with your interface.

And it's true. In this example, the first version represented the code we wanted to write. We only modified it, changing the interface in the process, to make it easier to test. Not only does reading the code now give us a distorted view of its use, but tools that use introspection (like automated documentation tools) will also pick up on the changed API.

A standard Python way of testing the first version of the code is to mock out OtherClass at the module level. Because names are resolved dynamically, instead of being statically bound, if we patch the class for the duration of the test then our mock will be used when we call some_method. Unfortunately the code to do this requires a bit of ugly and tedious boilerplate:

Most mocking frameworks are based on the 'record - > verification' pattern for testing. Not only do I find this backwards (I prefer the 'action -> assertion' pattern) but it means you often have to record every step in a test, when you are only interested in testing a corner case - one part of the unit behaviour.

Mock is a very lightweight mocking framework for unit testing. It also includes a decorator that makes it trivial to mock out classes in a module without the boilerplate.

The patch decorator monkey patches SomeModule.SomeClass only within the scope of the test and passes in the mock it creates to the test method. Actually the example above uses a couple of nice features of Mock (and its patch decorator) that aren't quite available yet. The code doesn't look very different with the current version though.

Of course in Python, modules are first class objects and are effectively singletons. This is one reason why the singleton pattern is so rarely useful in Python. It also means that changing module level attributes for tests is mutating global state. Forgetting, or botching, restoring state after your test will leave your patch in place for subsequent tests and cause you some very weird problems. This is reason enough for some people to say that it should never be done. One real consequence is that you can't run tests in parallel - but whilst running tests in parallel is fine, would you really do it using several threads rather than several processes? That's a recipe for disaster all of its own... Using patch ensures that the patching is undone whether the test passes or fails.

I originally created the Mock module to make some common testing patterns we use at Resolver Systems simpler. Since then we've made several improvements to Mock that have never been synced back in (plus I've had patches and feature requests from other users of Mock):

• A patch (1 line!) from Kevin Dangoor for nose compatibility (preserving line numbers in patched methods)
• Add the assert_called_with method
• Add a side_effect attribute - a callable used every time a Mock is called. This allows you to do nice things like raise an exception if a mock is called, or return members of a sequence for repeated calls rather than just a fixed value.
• Make the default return value a Mock (unless None is explicitly specified)
• Modify patch to allow a single string argument specifying what should be patched

Apparently this is my 1000th blog entry (on this blog). The first was on January 23rd 2005, which makes for an average of more than five entries a week over three and a half years! [1]

Another project I omitted from my list of projects I'm interested in was Tweezer. Tweezer is a very nice looking Twitter client by Ed Leafe, built on the Dabo framework. It's still missing some features, but I'm looking forward to hacking on it...

[1]	Over the last year I've also averaged more than five entries per week on the IronPython URLs blog, but as that is a link and news aggregator they tend to be a lot shorter.

---

Try Python (Python in the Browser) and Silvershell

One of the longer term projects I'd like to work on is the Try Python tutorial. The website currently just shows Python in the Browser, which is an interactive Python interpreter running in an HTML textarea on the webpage. The nice thing is that it is possible to preload samples into the interpreter and use it as live examples for a Python tutorial. This would be a great way to teach Python, requiring only a browser with Silverlight 2 [1].

Creating an effective tutorial is actually quite hard work, so it would be nice to build off an existing project. Crunchy is a related project that includes some tutorials already, but the 'textarea' type approach looks closer to the real Python interpreter than the single text entry field that Crunchy currently uses. I wonder whether integration with Crunchy would be an effective way to go - I really want to spend my time on making the tutorial effective rather than working on the implementation details of the interpreter.

That aside, "Python in the Browser" is currently not as visually appealing as it could be... There are at least three different ways for me to remedy this.

1. Borrow the Javascript from Jimmy Schementi's Try IronRuby.
2. Borrow the Javascript from Why the Lucky Stiff's Try Ruby.
3. Rebuild on Silvershell.

I've had permission for both 1 and 2, but Silvershell is a relative newcomer to the game. It is the creation of Dan Eloff, and rather than using the browser DOM and Javascript for the UI it uses Silverlight itself.

As you can see it is very visually appealing. It also has some nice features like auto-indentation and completion (neither of which would be particularly difficult to add to PiB). The only difficulty is that it might make it harder to simulate several interpreters in a page as it certainly used to be difficult to relocate the Silverlight control in the page [2]. Dan suggests we could solve that by locating the control in an iframe (my instinct was to spell that iFrame...) and relocating that.

Anyway it will be fun to experiment and I hope to end up with a useful set of resources for learning Python online.

[1]	And once Silverlight 2 is out of beta it will almost certainly be pushed out over Windows update like Silverlight 1. Not long ago Silverlight 1 downloads were running at a million a day [citation needed] so I don't expect deployment to be an issue.

[2]	Well, it used to hard crash the browser anyway...

---

Too Many Projects, Too Little Time

One annoying thing about spending the last year writing, is that I haven't been able spend much time on other projects (although I did recently manage to push out a ConfigObj release).

I've already spent (in my mind) the spare time I will have once the book is done three times over. Projects I have waiting in the wings include:

• ApiDoc

  This is a Python documentation tool that Resolver Systems uses to auto-generate our Spreadsheet API Documentation from ReST docstrings. I've had permission to release this as Open Source, at least partly because I wrote half of it on my own time one Christmas. It's features (which are the reason we wrote a new documentation tool) are:

  • Runs under IronPython and will generate docs for code written in IronPython (using introspection)
  • Allows you to specify which parts of your API should be public
  • Can raise an error if any public class / method / property does not have a docstring
  • It is very small (only a few hundred lines of code) and easy to customize
  • Fully tested of course

• Python-system

  This is (or will be) a compatibility layer for the .NET base class libraries so that IronPython code can be run under CPython. This will be useful for me if we make a source release of the Resolver One Library packages. This will allow you to integrate Resolver One spreadsheets, or even the whole calculation engine, into standard Python applications.

  As we haven't yet made a source release, it isn't the highest priority.

• DynamicScriptControl

  This project was started by Ivan Porto Carrero, the author of IronRuby in Action. It allows you to create user controls in XAML, for Windows Presentation Foundation and Silverlight, that are written in DLR languages (using normal XAML syntax for setting properties).

  Ivan has got it working with IronRuby and I've promised to get it working with IronPython.

• IronEditor

  An IDE for Dynamic Language Runtime languages (IronPython & IronRuby), compatible with both Mono and .NET. Ben Hall has made a great start on this project. I'm not sure how much time I will have to contribute to this, but I'm watching it with great interest.

In fact there are several more (well three...) that deserve blog entries of their own.

One final one that I might take on is a report generation tool for Resolver One. My Dad works in Industrial Safety and Reliability Engineering and often has to produce hazard reports. The standard tools are custom applications (inflexible and unfamiliar to users) or Excel (not really suited to the purpose but much more flexible than the custom applications). If I can create a report generation library for Resolver One that produces Word documents, then not only will it be useful for me but I can build a tool for him that is as flexible as Excel but customized for the hazard reports he needs to produce.

---

New Release: ConfigObj 4.5.3

A new version of ConfigObj is now available, version 4.5.3.

This version is a minor bugfix release. It fixes a relatively obscure bug, where an exception could be raised when validating a config file with copy=True and __many__ sections [1].

• ConfigObj 4.5.3

ConfigObj is a Python module for the simple reading and writing of config files. It has many features, whilst remaining easy to use.

With the assistance of validate it can validate a config file against a specification, and convert members to the expected type.

Eggs for Python 2.4 & 2.5 are available from Python Package Index.

[1]	The bug only showed up when validating a config file with empty sections validated by __many__.

---

Python in the Browser: Live Interactive Interpreter in an HTML Textarea

A while ago I experimented with an interactive Python interpreter that ran in the browser using Silverlight. Unfortunately a couple of bugs with the version of IronPython for Silverlight prevented it from working properly.

Now that Silverlight 2 Beta 2 is out, along with an updated version of IronPython for Silverlight, those bugs have been fixed.

"Python in the Browser" is an interactive Python interpreter that runs in the browser, using Silverlight 2 and IronPython.

• Python in the Browser Project Page

This is ideal for tutorials and documentation, where example Python code can actually be tried in the browser.

You can see a live demo of the current version at:

• Live Demo of Python in the Browser

It requires Silverlight 2 Beta 2, and the Python version is 2.5. Builtin modules are available, but I haven't made much of the standard library available in the demo (they simply need to be added to the 'xap' file containing the silverlight application.)

Although the version in the repository works, it has some known limitations. See the issues page for the known ones so far.

The interpreter runs in an HTML textarea, with Javascript that communicates with Silverlight and prevents you deleting text from the console except after the interactive prompt.

Target browsers are Firefox 2 & 3, Safari and IE 7 & 8. (It won't work in other browsers until there is a version of Silverlight that works with them.) Silverlight 2 is currently only available for Windows and Mac OS X, with the Firefox, Safari or IE browsers. Linux support is in the works via the Moonlight project from Mono.

The project is a combination of IronPython (for the interpreter loop), Javascript (for the 'console behaviour' in the textarea) and C# (as a helper to call into Silverlight from Javascript). On every keypress Javascript calls into IronPython (via the C#!). If the keypress is an 'enter', then it pushes the current line into the interpreter loop (which uses the standard library code module). Stdout is diverted to print into the textarea, where tracebacks are also sent. If you are attempting to type over, or delete, previous output then the keypress is cancelled.

---

Ironclad 0.4 Released: Use CPython Extensions from IronPython

William Reade of Resolver Systems has announced a new release of our Ironclad project.

• Ironclad Home Page
• Download Binaries and Source for Ironclad
• Ironclad Mailing List

Ironclad is a project to enable you to use CPython extensions implemented in C seamlessly from IronPython. This basically involves re-implementing the Python C API in C# (along with reference counting and garbage collection strategies that span the native code in the C extension and the managed code that lives in the .NET Virtual Machine).

This release comes a bit sooner than expected as William has got the import hook working. This means that after executing import ironclad in IronPython (2), you can import any C extensions on your path just as you would from CPython!

Here's the details from the announcement:

Ironclad v0.4 has now been released. Properties and (some) members on unmanaged types should now work, and it's now much easier to use -- by starting ipy.exe in Ironclad's build directory, you should now be able to 'import ironclad' and then attempt to import CPython extensions as you would in CPython. Note that they aren't likely to work very well yet, if at all, because so much of the CPython API is still unimplemented.

Our intention is that the next release, v0.5, should allow people to import numpy and do something incredibly trivial with it. It may be some time before we understand numpy well enough to achieve this, so we aren't in a position to commit to a release date; but we'll provide updates as we progress.

---

Open Source Code Review Tool: Rietveld

In November 2006, Guido van Rossum gave a public talk at Google about Mondrian, a web tool for code review on the web that he was developing at Google. He was hoping to release it as an Open Source tool, but due to its popularity within Google it became to closely tied to Google's internal infrastructure to disentangle.

He recently announced a new tool, for Subversion and based on the Mondrian code base but for the Google App Engine. As well as a powerful code review tool, it also serves as an example Google App Engine project. The online tool, which Guido recently invited Python developers to try, is called codereview:

http://codereview.appspot.com

This has now been released as an Open Source project with the name Rietveld:

• Rietveld Project Page
• Rietveld Mailing List

The name is in honor of Gerrit Rietveld, Dutch architect and furniture designer.

People are already making lots of feature requests, and now that reitveld is Open Source, they have the opportunity to contribute those changes...

---

Ironclad Overview

Ironclad is an MIT licensed Open Source by Resolver Systems. The goal of Ironclad is to allow you to seamlessly import C extension modules (for CPython) in IronPython.

• Ironclad Homepage, Download and Subversion Repository

The short version of how it works, is that Ironclad creates a fake Python dll, with the C-API pointing to functions mainly implemented in C#. These create and manipulate IronPython objects.

Resolver Systems has had one pair programming on the project (led by one of our core developers, William Reade) for several months. The current status of the project is that, for simple modules, module initialisation works. In the 0.1 binary release you can call functions. The version in the SVN repository allows you to use classes defined in extensions (so long as they only call the parts of the C-API that we have implemented). The file type kind-of works - you can open and close files but not yet read or write from them.

Here is a longer (but still brief) overview of Ironclad that I presented at the Python and .NET open space at PyCon.

We generate a stub-dll that looks superficially like Python25.dll. Prior to importing a CPython extension we load this dll into our process space and initialise it by passing in a pair of function pointers (that point to managed code). The dll initialisation function calls these repeatedly passing in symbol names (CPython's exported symbols - the CPython API). This creates the CPython API but with these functions pointing to managed code (delegates) that actually provides the implementation (in C#).

You can now load C extension modules ('.pyd') by creating a 'PydImporter' and calling its load method with the path to the binary on the filesystem. This loads the binary into the process space and calls the module initialisation function (e.g. initbz2 or initmultiarray). This initialisation function will call the C-API that we have already initialised.

Generally this will start off by calling 'Py_InitModule4'. One of the parameters to this function is a pointer to an array of PyMethodDef structs which defines the functions exported by the module we are loading. We generate IronPython code corresponding to these functions and execute it in a new module - to expose these functions to IronPython. These functions call delegates which point to the corresponding unmanaged code (the module!). We do a similar thing for the classes, which are added by calls to 'PyModule_AddObject'.

Almost all of the hard work is done by one monster C# class that actually implements the Python C API (or the parts of it that we have got to) - the 'Python25Mapper' (which inherits from 'PythonMapper' which is autogenerated).

When we call a function from IronPython - we call the mapper's 'Store' method on both the args tuple and the kwargs dictionary - this creates pointers representing the original items, which are then passed into the delegate. Marshally magic happens and the unmanaged function is called with pointers to args (tuple) and kwargs (dict).

(Inevitably the first thing that a function does is call PyArg_ParseTuple[AndKeywords]. This was so complicated that we actually just lifted the CPython implementation rather than re-implementing in C#.)

Eventually it will return something. This something will usually have been created through the C-API (so we have control over what was created) - so the PythonMapper has a reference to it, mapping a pointer to a managed object. When this pointer is returned (from the delegate) we check for errors set on the PythonMapper, raising the exception if necessary. If not we can 'Retrieve' the managed object and return it from the function.

An interesting point is that we have to handle reference counting. When a object is stored for the first time, the mapper allocates some memory following the layout of the normal 'PyObject' and sets the refcount to 1. Subsequent calls to 'Store' for the same object will increment the ref-counter and return the same pointer. When the refcount drops to 0 (as a result of managed or unmanaged code) this memory can be deallocated because we know that the unmanaged code has no references to.

Strings and tuples we need to handle slightly specially because C extensions need direct access to memory to some of their items. We do this by converting these objects on the way in (and on the way out if necessary). (Lists will need to be handled very specially at some point - but not yet!!)

In a call to a module function, once we have retrieved the result from the result pointer it is safe for us to DecRef the args and kwargs and result pointers.

Types are complicated, but fundamentally the same!

BZ2 is our test case so far. compress and decompress works. The BZ2Compressor and BZ2Decompressor types both work. BZ2File (passing file stream descriptors from managed to unmanaged code) is proving fun... (You can open and close the files, just not read or write anything yet.) Numpy next...

We haven't done much with the GIL yet!

The stub-dll actually uses assembly language to write a table of function pointers. Most of these functions are not implemented in C (except a few like PyArg_ParseTuple[AndKeywords]) but in C#. The assembly is a chunk of code with a series labels (function name) followed by a jump instruction (using a pointer to a managed delegate which calls into our C#). We do this in assembly language because gcc won't let us define C functions without a calling convention (it won't let us use __declspec(naked)).

So, lots of magic and lots of fun difficulties, but it is working so far! The binaries are only suitable for Windows and we haven't yet tried this project with Mono. William deliberately picked gcc as the compiler so that it could be ported to Mono though. There is a longer, more detailed, version of this overview in code repository.

The business value for Resolver Systems in this project is that our customers will be able to use Scipy in Resolver One, our IronPython spreadsheet.

There are two interesting potential 're-uses' of Ironclad.

• Some of the PyPy folk are interested in looking to see how much of the project they could reuse to allow you to use C-Extensions from PyPy.
• By rewriting parts of the top layer (or even just embedding IronPython) it could allow you to use Python C-extensions from any .NET language.

---

Release: ConfigObj 4.5.2 and validate 0.3.2

There was a bug in release 4.5.1 of ConfigObj, so there is now an updated release:

• ConfigObj 4.5.2
• Validate 0.3.2

The bug affected the use of None as a default value in configspecs and is now fixed.

What are ConfigObj and Validate?

ConfigObj is a simple to use but powerful configuration file management library. Features include:

• Nested sections to any depth

• Unicode support

• List and multiline values

• Integrated with a powerful validation system

  • including automatic type checking/conversion
  • repeated sections
  • and allowing default values

• All comments in a file are preserved when writing

• The order of keys/sections is preserved

• A powerful unrepr mode for storing basic datatypes

validate is the module (optional) used for config file validation and type marshalling. It can be used outside of ConfigObj for matching string values against a text specification which also does type conversion.

Changes in ConfigObj 4.5.x

Distribution now includes validate 0.3.2 so None as a default value in configspecs works again.

BUGFIX: Unicode configspecs now work.

ConfigObj will now guarantee that files will be written terminated with a newline.

ConfigObj will no longer attempt to import the validate module, until/unless you call ConfigObj.validate with preserve_errors=True. This makes it faster to import.

New methods restore_default and restore_defaults. restore_default resets an entry to its default value (and returns that value). restore_defaults resets all entries to their default value. It doesn't modify entries without a default value. You must have validated a ConfigObj (which populates the default_values dictionary) before calling these methods.

BUGFIX: Proper quoting of keys, values and list values that contain hashes (when writing). When list_values=False, values containing hashes are triple quoted.

Added the reload method. This reloads a ConfigObj from file. If the filename attribute is not set then a ReloadError (a new exception inheriting from IOError) is raised.

BUGFIX: Files are read in with 'rb' mode, so that native/non-native line endings work!

Minor efficiency improvement in unrepr mode.

Added missing docstrings for some overridden dictionary methods.

Added the reset method. This restores a ConfigObj to a freshly created state.

Removed old CHANGELOG file.

Changes in Validate 0.3.2

BUGFIX: None as a default value handled correctly.

BUGFIX: Unicode checks no longer broken.

Improved performance with a parse cache.

New get_default_value method. Given a check it returns the default value (converted to the correct type) or raises a KeyError if the check doesn't specify a default.

Added 'tuple' check and corresponding 'is_tuple' function (which always returns a tuple).

BUGFIX: A quoted 'None' as a default value is no longer treated as None, but as the string 'None'.

BUGFIX: We weren't unquoting keyword arguments of length two, so an empty string didn't work as a default.

BUGFIX: Strings no longer pass the 'is_list' check. Additionally, the list checks always return lists.

A couple of documentation bug fixes.

Removed CHANGELOG from module.

---

Release: ConfigObj 4.5.1 and validate 0.3.1

After one year and two days since the last release, there is a new release of ConfigObj.

• ConfigObj 4.5.1
• Validate 0.3.1

This release adds a few new features, plus has several bugfixes and minor performance improvements. Thanks to all those who gave feedback, reported bugs and provided patches.

What are ConfigObj and Validate?

ConfigObj is a simple to use but powerful configuration file management library. Features include:

• Nested sections to any depth

• Unicode support

• List and multiline values

• Integrated with a powerful validation system

  • including automatic type checking/conversion
  • repeated sections
  • and allowing default values

• All comments in a file are preserved when writing

• The order of keys/sections is preserved

• A powerful unrepr mode for storing basic datatypes

validate is the module (optional) used for config file validation and type marshalling. It can be used outside of ConfigObj for matching string values against a text specification which also does type conversion.

Changes in ConfigObj 4.5.1

Distribution updated to include version 0.3.1 of validate. This means that Unicode configspecs now work.

ConfigObj will now guarantee that files will be written terminated with a newline.

ConfigObj will no longer attempt to import the validate module, until/unless you call ConfigObj.validate with preserve_errors=True. This makes it faster to import.

New methods restore_default and restore_defaults. restore_default resets an entry to its default value (and returns that value). restore_defaults resets all entries to their default value. It doesn't modify entries without a default value. You must have validated a ConfigObj (which populates the default_values dictionary) before calling these methods.

BUGFIX: Proper quoting of keys, values and list values that contain hashes (when writing). When list_values=False, values containing hashes are triple quoted.

Added the reload method. This reloads a ConfigObj from file. If the filename attribute is not set then a ReloadError (a new exception inheriting from IOError) is raised.

BUGFIX: Files are read in with 'rb' mode, so that native/non-native line endings work!

Minor efficiency improvement in unrepr mode.

Added missing docstrings for some overridden dictionary methods.

Added the reset method. This restores a ConfigObj to a freshly created state.

Removed old CHANGELOG file.

Changes in Validate 0.3.1

BUGFIX: Unicode checks no longer broken.

Improved performance with a parse cache.

New get_default_value method. Given a check it returns the default value (converted to the correct type) or raises a KeyError if the check doesn't specify a default.

Added 'tuple' check and corresponding 'is_tuple' function (which always returns a tuple).

BUGFIX: A quoted 'None' as a default value is no longer treated as None, but as the string 'None'.

BUGFIX: We weren't unquoting keyword arguments of length two, so an empty string didn't work as a default.

BUGFIX: Strings no longer pass the 'is_list' check. Additionally, the list checks always return lists.

A couple of documentation bug fixes.

Removed CHANGELOG from module.

---

There was briefly a 4.5.0 release. Whilst updating the Sourceforge page I discovered a bug report and decided to fix it and do a new release before making the announcement. Sorry for any confusion this may have caused.

---