Python Programming, news on the Voidspace Python Projects and all things techie.

Job Hunting

emoticon:clock I haven't done much coding this week. Over the weekend I pushed out ConfigObj 4.2.0 (over 100 downloads after 3 days) and Movable Python Demo (around sixty downloads so far) releases. I have however, made some progress on the job hunting front.

I've been looking for programming work four days a week (I want to continue working at the Jesus Centre one day a week), in the Northampton area. This means that I'm expecting it to take a long time. Based on my experiences so far, that expectation may not be disappointed. Confused

As I don't have commercial experience, and my only programming skills (though excellent of course) are with Python, I've been looking for junior programming positions. I'm expecting to end up developing with PHP or ASP, although virtually any platform/language except Java or Perl would be acceptable.

These sorts of jobs only occasionally come up in the Northampton area. I have however found another type of job that seems to come up a lot more often. Some of these jobs would be a useful sideways step towards programming work. This is working doing IT support, usually in a helpdesk environment. Whilst it's not what I want to do, a lot of these jobs involve working with technologies like SQL, javascript, supporting web application development, XML etc. Commercial experience of any of these will make it much easier for me to get the kind of job I want.

Despite my negativity, it hasn't been an entirely fruitless search for programming work. In fact I may be facing something of a dilemma. Neutral

I really didn't want to work in London. I'm not willing to move, and the thought of a commute doesn't exactly fill me with joy. However, a vacancy has come to my attention that would be more than ideal. It involves working with a small firm in London, developing applications using IronPython for .NET. How cool is that. Surprised

I've had a phone interview with them, they seem to basically like me, and I've got a full interview coming up soon. Who knows, it's not impossible that someone will be paying me to hack on Python.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-02-24 09:47:01 | |

Categories: , ,


SlippedStream

emoticon:slippedstream I'm pleased to be able to announce that a new program SlippedStream is available from the Voidspace Shop. Turn your Windows PC into a multimedia workstation. :-

Scam free TV on your pc - 100% Free, 100% legal.

The SlippedStream desktop application gives you the best Sky, TV and Radio stations on your pc - just install SlippedStream and view at your leisure.

This unique application has a stylish design and over 100 Sky and TV Channels including countless live radio stations categorised by type and with easy installation.

SlippedStream brings you the Sky and TV channels and programmes that are worth watching and listening to and compiles them to a handy desktop application - 'SlippedStream'.

Now includes such content as Simpsons, 24, Lost, Family Guy, King of the Hill, Viva La Bam (many of which are whole entire seasons if not multiple whole entire seasons), AND top movies such as Troy, Robots, Braveheart, Italian Job and more!

Scam free

Firstly, let's just say that this is scam free. Be assured that this isn't just a load of broken link trawled from the internet or a circuit diagram on how to break your digi box. This is a genuine product developed using Windows Media Player, Winamp, Quicktime and Real applications to give you what you always wanted - TV on your PC!

What is it?

SlippedStream is a desktop mini-application that provides you with easy access to legal and freely available web tv feeds that can be viewed by anybody provided they have the correct media player and/or plugin installed.

If this is just stuff from the web, why pay?

Good question. Mainly because we have searched out, tested and amalgamated these channels into a viable product. We have scoured until the ends of the web until we were sick of searching. We have probed various media streams, extracted hidden urls and tested them until we can't take it anymore. And then we made this nice little mini-application so you get all we have strived for via a simple click!

Safe

SlippedStream contains no spy-ware, viruses, or other malicious applications that silently attack your computer while you are surfing the Web and is NOT advert supported..

What you do get

Over 100 quality channels of all genres: Movies, science, news, entertainment, fashion, shopping, cartoons, sport, latest TV episodes, comedy and more!

SlippedStream is LEGAL therefore doesn't carry channels that can only be received through subscriptions BUT you will be surprised at the quality you can receive.

Name drop

Ok, you want a name drop to get you excited? How about this:

Exclusive content featuring, MTV, Extreme sports, BBC, CNN, VH1, Movies, Cartoons, stacks of radio station, and on demand programmes.

Also top series such as Simpsons, 24, Lost, Family Guy, King of the Hill, Viva La Bam (many of which are whole entire seasons if not multiple whole entire seasons), AND top movies such as Troy, Robots, Braveheart, Italian Job and more!

What you won't get

We don't need to pretend, you WON'T get full blown, dvd quality SKY broadcasting that would normally cost you $XXX. If we said we could we'd be lying or it would be ILLEGAL (and a scam).

You won't get top quality SKY BOX OFFICE for example. No one can get you such things except SKY.

Most of all we're not selling you just some Peer2peer software for $XXX claiming that you're buying a download subscription with legal unlimited movie downloads per year (p2p software IS legal, downloading anything using it IS NOT - making it useless).

No hardware

No special hardware is required to use the application. Further software installation may be required but it is all free and legal.

No licence

Since you are not using special receiving equipment or TV cards, NO TV LICENSE is required and there are NO SUBSCRIPTIONS needed.

Who needs SlippedStream?

Those of you who will find it very cool to have internet TV at the click of the mouse on your PC.

Those who can sit at work and watch cartoons, movies, etc.

Those who want to check news or weather reports live throughout the day.

I could go on...

Satisfied Customers

SlippedStream is currently selling at more than ONE COPY PER DAY. And that's to satisfied customers.

Requirements

  • Internet Connection (preferably Broadband).
  • Recommended desktop resolution of 1024x768.
  • Windows Media Player, Winamp, Quicktime and Real applications (all free downloads).

Note

This is not a Python program, and is currently Windows only. A Python version, which will work on Linux and Mac desktops, will hopefully follow shortly.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-02-24 09:09:04 | |

Categories: ,


ConfigObj & Movable Python: Oops!

Embarassed Their are minor mistakes in the distributions of both ConfigObj 4.2.0, and Movable Python.

When I released the finalised version of ConfigObj 4.2.0, I forgot to bump the version number in the source code from 4.2.0beta2, to 4.2.0. This is now corrected, you may or may not want to download the corrected version.

In the Movable Python distribution I forgot to include roman.py, which is needed by docutils. My apologies. You can download it and just drop it into the lib/ directory.

Update

My mistake - looks like roman.py is included in the distribution of Movable Python. The user error report was actually about the installation process of docutils on the Mac, not about Movable Python. Smile

Whilst we're on the subject of distributions, it looks like Stewart Midwinter has stabilised his first set of changes to Firedrop2. A new release should follow in the next few days. It's quite a list of changes ! Expect more progress to follow. Smile

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-02-24 08:55:19 | |

Categories: ,


Python 3000, Files and Text Encodings

emoticon:pill As I'm sure you're aware, Guido has often pronounced that there will never be a Python 2.10 [1]. We're currently on Python 2.4 and momentum is building for Python 2.5. This leaves a finite, and ever dwindling, amount of time before Python 3.0 must be upon us.

In the past this has seemed such a distant possibility that it is jokingly referred to as Python 3000, or Py 3k for short. The 3000 being the year it was expected to actually arrive...

Py 3k will be an especially significant version of Python because Guido will allow it to break backwards compatibility. At last various warts and inconsistencies in the Python language will be removed. The downside is that a lot (probably most) legacy code will break, possibly never to traverse the great divide that this little version increment signifies.

A few of the major changes on the cards include :

  • Old style classes will vanish forever. All classes will then be new style classes.
  • Integer division will return floating point results. [2]
  • Print will become a function. [3]
  • All strings will be Unicode objects. Binary data will be stored using a new bytes builtin type.

Interestingly, after much argumentification, lambda will stay.

Frighteningly enough, this is no longer hypothesis and wild conjecture. Guido has recently jumped into the loving arms of google, who have freed him up to work 50% of his time on Python. This is great news. In the last couple of weeks (on Python-dev) there has been a veritable clamour of debate on the semantics and even implementation details of Python 3000. The new bytes builtin [4] is even scrambling to see if it can make it in to Python 2.5 which will happen this year.

After a long rambly introduction, it is the last change of my list that the rest of this entry is about - strings and Unicode.

Arguably, the default behaviour of Python for reading files is broken. The following basic line of code works perfectly for reading binary data or text on a Lunix type platform :

data = open('filename.dat').read()
# ...code that modifies data...
open('filename.dat', 'w').write(data)

This is such a mind-numbingly obvious idiom that it scarcely warrants mention; aside from the minor fact that if you move this code to Windoze (and you are reading binary data) it breaks.

When reading the data, any occurrences of \r\n are silently converted to \n. On writing, \n becomes \r\n. For text this is what you want (and very useful), but for binary data it corrupts it.

This of course is a minor wart, and although it bites most newbies at some point, it's easily solved by using binary mode.

data = open('filename.dat', 'rb').read()
# ...code that modifies data...
open('filename.dat', 'wb').write(data)

Nonetheless, the default mode is not binary mode and coders for the Linux platform are that bit less likely to write platform portable code. (Darn finger typing for the sake of a platform that's already broken, grumble, grumble...)

What's more serious is that probably most programmers avoid Unicode like the plague. Smile

It's much easier for us Western European and other English speaking programmers to just assume that all text is encoded in Latin-1 (or some other ASCII compatible encoding) and only ever use the str object.

Note

Python actually makes using Unicode fairly straightforward once you understand the basic issues.

For a great introduction, may I humbly recommend A Crash Course in Character Encoding.

There is an error in the code there by the way, it uses codecs.BOM_UTF7, which of course doesn't exist. Unfortunately PyZine aren't updating any more, so it can't be fixed. Sad

This of course is a fundamentally broken approach. Surprised

If you don't explicitly know the encoding of text represented as a byte-string, then effectively you have random binary data rather than text. Unfortunately it's an approach that 'just works' most of the time, but causes horrible problems when it doesn't.

Python 3000 is going to simplify this for the majority of cases. All string instances will be unicode. Binary data (as mentioned before) will use the bytes type.

This means that when you open a file in text mode, a new I/O layer will determine the encoding and return a unicode object [5].

When you open a file in binary mode, you will get back a bytes instance.

So how does the I/O layer determine the encoding of a piece of text ? Guido says that it will do it in the same way that programs like notepad do. In other words, it will check for the existence of a UTF8/UTF16 BOM, and if this isn't present it will assume that the text is encoded in the default way for the users locale.

If you know the encoding you will be able to specify it when you open a text file.

This means you must either know the encoding of the text, or hope that Python gets the right encoding for you. Obviously knowing the encoding is preferable, but what do you do if you don't know ?

This is the problem I face in rest2web. It reads in source text files, and outputs target HTML in a consistent encoding. My source files come from a variety of places, and unless I created the text myself I rarely know the encoding used.

There are various heuristics you can use to make an educated guess at the encoding in use.

The basic technique is to check for the existence of a UTF8 or UTF16 BOM. If this is absent, you can use the locale module to determine the standard encoding(s) for the user's locale. You can then try to decode the text to Unicode, using these encodings (along with a couple of common ones) one by one. You can then assume that the first decode which works is the right one.

Here is an example function which does this (taken from rest2web, but borrowing heavily from code in the docutils project) :

import locale
import codecs
encoding_matrix = {
   codecs.BOM_UTF8: 'utf_8',
   codecs.BOM_UTF16: 'utf_16',
   codecs.BOM_UTF16BE: 'utf16_be',
   codecs.BOM_UTF16LE: 'utf16_le',
}
def guess_encoding(data):
    """
    Given a byte string, guess the encoding.

    First it tries for UTF8/UTF16 BOM.

    Next it tries the standard 'UTF8', 'ISO-8859-1', and 'cp1252' encodings,
    Plus several gathered from locale information.

    The calling program *must* first call
        locale.setlocale(locale.LC_ALL, '')

    If successful it returns
        (decoded_unicode, successful_encoding)
    If unsuccessful it raises a ``UnicodeError``.
    """

    for bom, enc in encoding_matrix.items():
        if data.startswith(bom):
            return data.decode(enc), enc
    encodings = ['ascii', 'UTF-8']
    successful_encoding = None
    try:
        encodings.append(locale.nl_langinfo(locale.CODESET))
    except AttributeError:
        pass
    try:
        encodings.append(locale.getlocale()[1])
    except (AttributeError, IndexError):
        pass
    try:
        encodings.append(locale.getdefaultlocale()[1])
    except (AttributeError, IndexError):
        pass
    # latin-1
    encodings.append('ISO8859-1')
    encodings.append('cp1252')
    for enc in encodings:
        if not enc:
            continue
        try:
            decoded = unicode(data, enc)
            successful_encoding = enc
            break
        except (UnicodeError, LookupError):
            pass
    if successful_encoding is None:
         raise UnicodeError('Unable to decode input data. Tried the'
            ' following encodings: %s.' % ', '.join([repr(enc)
                for enc in encodings if enc]))
    else:
        if successful_encoding == 'ascii':
            # our default ascii encoding
            successful_encoding = 'ISO8859-1'
        return (decoded, successful_encoding)

A couple of important things to note. In order to be effective, the code calling this function must first have imported locale and called locale.setlocale(locale.LC_ALL, ''). Because of the way this effects the underlying C access to locale information, this should only ever be done once by an application. This is why it is left to the code calling this function.

Secondly, on many platforms this code will attempt to decode with several 8 bit encodings (ascii, UTF8, Latin-1, macroman and cp1252 are all common). Unfortunately, text encoded with one of these may well decode successfully with one of the other encodings. Any bytes representing non-ascii characters are likely to still be valid in another encoding, but refer to the wrong character.

This means that the above function may return the wrong eight bit encoding. sigh

There is a further trick you can use to determine which 8-bit encoding is in use. This recipe from the Python Cookbook knows which bytes commonly appear in several 8-bit encodings and will 'guess' which one is in use.

Note

Python 3000 won't use any of these tricks, the I/O layer will just use the default encoding that the user has set. More than likely the user didn't set it, but uses the operating system default for the language chosen.

In summary, if you don't know the encoding of a text file it is a lucky dip as to whether you can correctly decode. This won't improve dramatically in Python 3000, but it will only be as bad as any other program you use.

[1]This is because he hates the ambiguity of double digit version numbers. Rightly so IMHO. A recent thread on Python-dev did suggest hexadecimal or roman numerals as a way round this. Wink
[2]Presumably, only if necessary.
[3]This fact alone could consign millions of lines of unmaintained code to the rubbish heaps of history.
[4]But not the all strings to Unicode change, although it was suggested as an optional switch.
[5]Instead of using open to open files, you may have the choice of two new builtin functions opentext or openbinary. This is still a matter of debate on Python-dev.

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-02-22 11:55:30 | |

Categories: ,


ConfigObj 4.2.0

emoticon:home ConfigObj 4.2.0 is now available.

The main improvements are full Unicode support, as well as the addition of the following 'convenience' Section Methods :

  • as_bool
  • as_int
  • as_float

More compelling reasons to choose ConfigObj for reading and writing config files in Python. Smile

What's New ?

The full changelog is :

Removed BOM_UTF8 from __all__.

The BOM attribute has become a boolean. (Defaults to False.) It can be True for the UTF16/UTF8 encodings.

File like objects no longer need a seek attribute.

Full unicode support added. New options/attributes encoding, default_encoding.

ConfigObj no longer keeps a reference to file like objects. Instead the write method takes a file like object as an optional argument. (Which will be used in preference of the filename attribute if that exists as well.)

utf16 files decoded to unicode.

If BOM is True, but no encoding specified, then the utf8 BOM is written out at the start of the file. (It will normally only be True if the utf8 BOM was found when the file was read.)

File paths are not converted to absolute paths, relative paths will remain relative as the filename attribute.

Fixed bug where final_comment wasn't returned if write is returning a list of lines.

Deprecated istrue, replaced it with as_bool.

Added as_int and as_float.

What is ConfigObj ?

ConfigObj is a simple but powerful config file reader and writer: an ini file round tripper.

Its main feature is that it is very easy to use, with a straightforward programmer's interface and a simple syntax for config files. It has lots of other features though. It is intended as a more powerful (but simpler to use) replacement for ConfigParser.

Its features include :

  • Nested sections (subsections), to any level

  • List Values

  • Multiple Line Values

  • Full Unicode support

  • String interpolation (substitution)

  • Integrated with a powerful validation system

    • including automatic type checking/conversion
    • repeated sections
    • and allowing default values
  • All comments in the file are preserved

  • The order of keys/sections is preserved

  • No external dependencies

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-02-19 17:12:26 | |

Categories: ,


Movable Python Demo

emoticon:movpy2 Hot on the heels of Movable Python 1.0.1, is the release of a free trial version of Movable Python.

This is the full distribution for Python 2.3.5. To download (and play with it), visit the Movable Python Demo.

Movable Python Demo Version

It is set to expire on the 22nd May, and displays a nag screen on startup. Other than that, it is the full version. Have fun. Smile

Like this post? Digg it or Del.icio.us it.

Posted by Fuzzyman on 2006-02-18 12:09:53 | |

Categories: ,


Hosted by Webfaction

Counter...