Python Programming, news on the Voidspace Python Projects and all things techie.

You Don't Really Need It

emoticon:waffle Python is my favourite language, by a very long way. That's not hard though, I've only done serious programming in Python, C# and Javascript and am familiar with the syntax (to lesser or greater degrees) of C, VB.NET and Ruby.

I program in the IronPython dialect of Python for my day job and for the book I'm still writing on, but CPython still has my heart. I love writing the examples for my book as I prefer coding to writing (writing still comes in a close second though), and I'm really looking forward to finishing the book as I will be able to get back to hacking on projects with CPython.

That aside, IronPython does have some advantages over CPython. The threading model in IronPython (based on .NET threads of course) is better than CPython, there is no GIL so multiple threads can scale across multiple cores and you can terminate threads. Additionally, extending IronPython from C# is worlds easier than writing C extensions for CPython. Writing C extensions for CPython may be relatively straightforward, apart from the usual issues with writing in C, but C# is quite a nice language - you still get the speed advantages, but you have a modern object oriented language with memory management. More importantly, IronPython can use .NET objects natively, so no need to use a C API to interact with Python objects [1].

Anyway, this is not at all what I intended to write about. Smile

In general the Python community is friendly and helpful, but there are certain topics that when people ask how to do them 'the community' [2] tends to reply with 'you don't really need to do that'.

Obviously in some circumstances that is the right response, but in other situations it seems to be a knee-jerk rationalisation - this is difficult or impossible to do in Python, therefore not wanting to do it is the right answer.

Topics that come under this category include:

  • Compiling as a way of obscuring source code.

    IronPython 1 can compile source to IL bytecode and Resolver One does use this to protect our source code - it isn't clear whether this will ever be directly possible with IronPython 2.

    Often this question is asked by newbies, where the "you really don't need to do that" response is appropriate - but not all reasons for wanting to protect your source code are nefarious. Razz I'm sure that Python has lost potential users due to the you're evil or stupid for even wanting to do this type responses that you sometimes see on comp.lang.python.

    The other response is "even if you compile down to obfuscated machine code you still haven't really achieved your goal because it's theoretically possible for someone to unpick that", which is one of those technically correct but not helpful or useful responses...

  • True concurrency (across multiple cores)

    The standard response in Python, is that if you need concurrency across multiple cores (or even multiple threads that really run concurrently) then what you really want is to run multiple processes. For some problems that is a really good answer, but for others it is a really bad answer (typically those that would require a lot of data or object graphs to be marshalled in and out of the multiple processes).

  • Terminating threads

    Doesn't come up very often, but Python doesn't let you do it because it doesn't think you really want to...

    See this short thread on threading or my article on threading in IronPython for some of the issues.

  • Private members

    See this thread on comp.lang.python. Smile

    This is something that isn't possible from Python, you can obscure things but not hide them altogether. And in general you don't need to anyway, certainly orders of magnitudes less than the proponents of typical statically typed languages would suggest.

    However, just because you don't need it very often doesn't mean that there are no possible good reasons for wanting it. I think that both sides of the debate (from the very long thread referenced above) are neatly summed up in:

    I understand the difference between the documented interface of a system and the details of its implementation. But sometimes it can be useful to take advantage of implementation details...

    By enforcing your 'data hiding', you're effectively telling me that I'm too stupid to make rational decisions of this sort. And that's actually extremely insulting. -- Mark Wooding

    By telling me that I can never have a valid reason for wanting 'data hiding' you're effectively telling me that I'm too stupid to make rational decisions of this sort. And that's actually extremely insulting. -- me

    The "programmers aren't stupid" philosophy has to cut both ways...

    Having said that, the argument that if it is present in the language it will be abused is compelling, that doesn't mean it can't be a problem occasionally.

UPDATE: The point of this entry is not to point out areas where CPython needs to change. Some changes are not realistic, and some (private members for example) are probably on balance a bad idea. The issue is that the community realise that some questions are not best answered by "you don't really want to do that", and that for some use cases CPython does have limitations.

In fact for several of these questions there are answers. For 'compiled distributions' you can just distribute '.pyc' files and this is often obfuscated enough for people's needs.

For private members / hidden data, you can write in C and not expose stuff. This isn't ideal of course, but it can be done.

The threading situation is more difficult. In the long run I would like to see (perhaps) PyPy become a viable alternative. In the meantime alternative implementations like Jython and IronPython don't have the same restrictions as CPython (but can have their own drawbacks of course - mainly lack of access to standard C extensions).

There's a good response from Tim Golden by the way: Even if you do need it.

I removed the reference to 'freetards' as I don't think it improved this entry. Smile For those unfamiliar with the term, it is used a great deal by Fake Steve Jobs and is a derogatory categorisation of those zealously enthusiastic about Open Source and free software. The irony of me using it, is of course that I'm as much a freetard as the next Python user...

[1]There are some details that help when writing class libraries in C# / VB.NET for use from IronPython, which is the subject of chapter 14 of IronPython in Action which I'm just finishing...
[2]A word which out of context means nothing of course...

Like this post? Digg it or it.

Posted by Fuzzyman on 2008-06-10 20:20:15 | |

Categories: , Tags: ,

Hosted by Webfaction