Python Programming, news on the Voidspace Python Projects and all things techie.
Sensible title this time - I've put a couple of tutorials online about http related subjects.
- A tutorial on using urllib2. It covers fetching URLs, coping with errors, and using handlers and openers. It should be useful to anyone new to using urllib2.
- A tutorial on BASIC authentication with Python. It first illustrates how BASIC authentication works - manually encoding the username/password into the request. Next we show the proper way to do it - with password manager, handler, and opener; a subject which has been known to confuse even intermediate level Python Programmers.
See : HTTP articles
Document Driven Design
Okay, so I'm cheating; the following words are my own, but I still think they're true. Documenting code is as important as testing (pauses and waits for flames).
"In my opinion, writing the documentation is an essential part of coding. In this phase you are forced to thing about how people might actually use your code. This inevitably exposes inadequacies and inconsistencies."
"At the other end of the process, starting to write your documentation as a spec can help you make implementation and design decisions. It also makes writing the docs after coding less hard work - this is especially important for Open Source projects where documentation is often an afterthought and not seen as part of the design process."
"It works like the old adage; describing your problem to other people often helps you find the solution. This is because you're forced to see things objectively. You're looking from outside the box... hopefully you take time to think there too ."
The latest update to rest2web has been ready for about two weeks now. In the process of documenting it I've made lots of changes, including a few bug fixes and reworking the way it does things. Unfortunately I also kept on having new ideas - which kept delaying it even further.
sigh This post has uncovered a limitation in the blog client which I need to fix. Where it puts the first part of an entry into the RSS feed, it can cause invalid markup. I think it needs to not render markup for the RSS feed. This would also get rid of the HTML in the RSS feed which looks really ugly over at PlanetPython.
UPDATE: The HTML in the RSS feed is from the macros being expanded in the wrong order, which I've corrected. The invalid markup was from one of my own macros and not the blog client.
I do wonder if I ought to have more descriptive entry titles, but it's so much more fun to be obscure.
Anyway - this is to announce a bugfix and update to the listquote module. Thanks to a fellow called Par Pandit for the bug report and fix.
listquote is my module for parsing strings into lists, even nested lists. It also properly handles quoting/unquoting of elements, comments on lines, building strings from lists, and even reading CSV files. Previous versions would truncate the last item of a list if it was empty.
i.e. lineparse(" 'member 1', 'member 2', '' ") became ['member 1', 'member 2']... oops .
It's fixed now with several optimisations  and another minor bugfix as well. The problem is that I have several modules relying on listquote - so I'm now recommending that users of the following packages download new versions from Voidspace :
- pythonutils module
The Pythonutils module also has updated versions of cgiutils and pathutils.
listquote Version 1.3.0 1st June 2005 Fixed bug in lineparse handling of empty list members. Thanks to bug report and fix by Par Pandit The 'unquote' function is now regex based. (bugfix it now doesn't return a tuple if fullquote is 0) Added the simplelist regex/function. elem_quote and uncomment use a regex for clarity and speed. Added a bunch of asserts to the tests.
|||Using regular expressions, so some of the functions should be quite a bit faster.|
The Second Coming
Well not quite... but rest2web 0.2.0 is now available from Voidspace.
See rest2web Homepage for download details and full documentation.
This is a major update with lots of new and funky features.
The main new features are :
- rest2web now renders pages a whole directory at a time and builds a tree of the site structure as it goes. This allows the new standard functions to provide very simple ways of automatically building sidebars. These data structures are also available for you to work with yourself.
- The new standard functions also include a one line way to print navigation trails.
- You can now specify your template encoding and output encoding - rest2web is fully unicode internally. It ought to 'do the right thing' if you don't specify what encodings are being used (even if you mix encodings).
- A new system of macros for frequently used shorthands (including smilies and Python source coloring).
- Lots more values available in the template namespace.
Grab it at rest2web Quick Download (725k)
rest2web requires the pythonutils module
What is rest2web ?
rest2web is a tool for automatically building websites, or parts of websites. It allows you to maintain your content as reST (see http://docutils.sourceforge.net) - a relaxed (WYSIWYG) text markup. (You can keep content as HTML as well if you prefer).
It will generate HTML from your contents and insert it into HTML templates, automatically building index pages, sidebars, and navigation trails (breadcrumbs) as it goes. It features a simple templating system, macros, and more.
Adding pages to your site can be as simple as dropping the text file of content into the desired directory. rest2web will add it to the index page and build the target file. If you delete a page, rest2web can automatically update the indexes for you.
Because rest2web resolves all links locally, it can be used to create HTML files viewable from the filesystem. This makes it an ideal way to create documentation.
See the docs and example site (included in the distribution) : rest2web Homepage
The example site illustrates most of the features of rest2web, which is now building a significant proportion of Voidspace - see the technology section for a live example.
What's Changed Since 0.1.0 ?
Version 0.2.0 2005/06/01 Various minor changes especially additions to the namespace pages are rendered in. Sites are rendered a whole section at a time. This means pages have index data for that section available. This is the ``sections`` variable in the namespace. Added the ``output-encoding`` and `final_encoding`` values to the restindex. Added the ``template-encoding`` value to the restindex. rest2web is now entirely unicode internally. It's now possible to specify title and description for the default section. Added indextree and thispage, allows building of sidebars. Added standard functions. Added macros. Started using subversion repository. Changed all line endings to 'LF'.
What's Next ?
The next step is to have rest2web automatically create contents and sitemaps for sites it is creating. I'd like to integrate with a system that inspects Python objects for helping to automate some of the process of documenting Python projects.
Spam, Spam, Spam
I'm back at work after a few days off ill... Some bizarre bug that went and then came back again. Four days offline. When I came in this morning I had 166 emails that had got past the spam filter (and countless more that didn't). After a bit of work (and some real work of course) I'm down to 21 that I need to deal with in some way.... sigh
During the time off I've cleaned up various things - so I'll do blog entries for all of them, by way of announcement. Unfortunately due to technical problems the projects will only be available for download from Voidspace for the moment, not yet from Sourceforge. Our sys admin at work is wrestling with an ipcop box and trying to get it to temporarily open ports apparently .
This work is licensed under a Creative Commons Attribution-Share Alike 2.0 License.