Python Programming, news on the Voidspace Python Projects and all things techie.
Mooching Books with BookMooch
I've been reading occasional entries on the Zen Habits Blog. One suggestion was that you can reduce your number of unnecessary possessions (something that resonates with me) by getting rid of books that you aren't going to read again. I've been wanting to try out BookMooch.com for a while and this gave me the push I needed.
BookMooch is a site for trading books. You keep an inventory of books you are willing to send to other people and a wishlist of books you want to receive. Sending books gains you points, and receiving them costs points. The site seems to have a critical mass of members that makes is worthwhile. Most books I am interested in, even if there aren't necessarily any copies currently available, have been traded several times. As a web based service it obviously has a high percentage of geek users, so even tech books seem to be pretty readily available.
Last weekend I listed about fifteen books I didn't need to keep, and by Monday five of them had been requested. Great.
So why is this on my techie blog?
Well I first came across BookMooch from some fliers at the last PyCon. I've also seen several references to the creator of BookMooch (John Buckman) giving talks at Python meetups. I can't find any references to what language / framework BookMooch is built with. Anyone know?
Of course BookMooch has an API if you want to play around with book trading / availability data. There are also frequent updates on the BookMooch blog listing new features, so BookMooch seems to be growing and improving fairly quickly.
The service is free, but makes money from Amazon affiliate links on all the books, which is a cool unintrusive business model for a useful (and environmentally friendly) website.
Game Development with Python and PyGame
I've just received a copy of Game Development with Python and PyGame by Will McGugan. I've only flicked through it so far, but it has really whetted my appetite to play around with PyGame (did you ever see those psyco presentations by Armin Rigo that use PyGame? Genius.)
Unfortunately it will probably be a while before I get a chance to properly review this, due to the minor issue of deadlines for my own book, but it is great to see a wider range of books exploring Python (there is even a Python for mobile devices with S60 book) coming out. Good luck Will!
Line Endings, Windows, Python and Text Mode
As everyone in the world surely knows, line endings for text are different on Unixy machines and Windows machines. Unix type systems use '\n' to represent a newline whilst Windows uses '\r\n'. This of course is a constant source of fun and joy to programmers who want to write code that will work unmodified across platforms.
Python attempts to abstract this difference away by saying that the canonical line ending for text within Python code is '\n'. Multiline string literals in sourcecode, will always use '\n' as the line ending. In addition, the default mode for reading and writing files is 'text mode'. In this mode Python claims to know about the line endings used by the underlying platform, and will write '\n' as '\r\n' on Windows and read '\r\n' as '\n'.
On Unix systems this means that strings are written out unchanged. This is the first cause of portability problems. If you are writing binary data (stored using the string type in Python 2.X) then code that uses the default (text) mode for files is broken on Windows. Binary data is very likely to contain bytes with a numerical value of ten (the ascii encoding of '\n'), and this will be written out as '\r\n' on Windows. Conversely, if the data happens to contain a '\r\n' the '\r' will be lost when the data is read in.
This is a well understood problem, and the solution is to always use binary mode for reading and writing binary file formats, even on platforms where it is technically not necessary.
The other side of the problem, is what happens when you write a string containing '\r\n' in text mode on windows. Python knows that the '\n' should be written as '\r\n', so what it actually writes is '\r\r\n'. As it happens, because the same mangling is done on the way back in, '\r\r\n' will then be read as '\r\n'. So all is well? Only if the generated file never has to be consumed by another program!
So how often does this happen? There was a recent discussion on the subject in Python-Dev. An IronPython user had been writing out data from a Windows Forms multi-line textbox, and was surprised to discover what Python had done to his text. Unsurprisingly, Windows Forms controls (which wrap native Win32 controls) return strings with Windows line-endings.
What did surprise me was a few people suggesting that IronPython ought to convert line-endings in the strings for you. Aside from thinking that having IronPython 'mangle' my strings for me behind the scenes is a truly terrible idea, this particular problem is not at all limited to working with Windows Forms from IronPython. As a general principle, Python doesn't attempt to protect you from platform differences but does expose platform functionality to you. As far as I know most of the major UI toolkits for Windows will also present newlines from user input as '\r\n' (certainly true for wxPython). This is because this is what Windows gives you when the user hits return inside a multiline text control. Some of the guys on Python-Dev considered it a bug that the Python wrappers don't convert these newlines into the Python canonical representation. I think that it is just something that you need to be aware of when working with Windows, and that if we were 'protected' from this platform difference it would have other unforeseen consequences.
It's not just something that happens with graphical programs. I was prompted to write this blog entry by a recent email on the Pylint mailing list:
The file produced by the command below contains non-standard line endings causing some editors (including MS Visual Studio) to show blank lines. The line endings produced by pylint contains two CRs instead of one (0D 0D 0A)pylint --generate-rcfile > pylintrc
I am using pylint 0.13.2 on MS Windows XP.
Not a big deal, but would be nice if this can be fixed at some point.
I'm not actually proposing a change to Python. I wouldn't be upset if in text mode '\r\n' was written as '\r\n' on Windows rather than '\r\r\n'. On the one hand I can't see that the current behaviour is ever useful, but on the other hand it is not a difficult problem to solve.
Whenever you receive user input that has come from a native UI control on Windows, you have to do some_string.replace('\r\n', '\n'). If you want your data to be displayed correctly, you have to do the reverse on the way out: some_string.replace('\n', '\r\n'). We do this in a few places in Resolver (where we use Windows Forms) and it is not very painful. It is however a common Python gotcha for Windows development, and worth reiterating the issues.
This work is licensed under a Creative Commons Attribution-Share Alike 2.0 License.