Python Programming, news on the Voidspace Python Projects and all things techie.
Interview with Pardus Linux
I recently did an interview on Python with the Pardus Linux magazine. Pardus Linux is a distribution developed in Turkey (by the Turkish National Research Institute of Electronics and Cryptology) with the goal of being usable by "normal" people rather than just geeks.
Pardus are great supporters and users of Python. A while ago they chose Python as their standard language for custom package and configuration management tools.
The magazine is Turkish, so here's the original in English. It covers topics like why use Python?, choosing a GUI toolkit, the move from Python 2 to 3 and whether there will ever be a Python compiler. The Turkish version is a very nicely produced PDF with waaay too many pictures of me, and even one of my wife.
Who is Michael Foord?
I've been developing with Python since 2002. I've written many articles on Python and maintain several open source projects. In 2006 I started working with a firm called Resolver Systems developing a highly programmable spreadsheet application with the then young IronPython. I wrote the book IronPython in Action with a colleague and was made the first Microsoft MVP (Most Valued Professional) for dynamic languages.
In 2009 I became a core Python developer, Python Software Foundation (PSF) member and part of the Python.org webmaster team. I've also been involved in helping to organise both PyCon in the US and PyCon UK (which in 2009 and 2010 is hosting EuroPython).
I'm now a freelance developer currently working with a German firm called Comsulting. We're developing web applications with Django, deployed on Linux of course, but with the client side written in IronPython on Silverlight.
I blog and write a great deal about Python and other technical topics.
In the real world I live in Northampton, in the UK, with my wife Delia.
Which department of Python development are you working?
I mainly help maintain the standard library, and in particular maintain the testing framework unittest. I've also helped with making parts of the standard library compatible with IronPython.
Why Python? What's the main idea of development of Python?
Python is a widely used programming language with a large(ish) code base. Parts of the code, in particular parts of the standard library are quite old. There are generally good tests, which are getting better, but we still get a lot of feature requests and bug reports. There is always more than enough work, and for me it is a privilege to be able to contribute back to the open source project that has given so much to me.
On a more self-interested note I am a big fan of the unittest style of testing. However, other testing libraries like nose and py.test have advanced the state of the art of testing in Python a great deal whilst unittest has stagnated. It has been great to get some of the tried and tested features from other testing libraries, like test discovery, into the standard library.
In the last few years Python has evolved a great deal, in particular the release of Python 3 which deliberately broke backwards compatibility with the Python 2 series to fix a number of longstanding 'warts' with the language. Python 3 is a great release, and comes with the full Python standard library, but third party libraries and the community are only gradually migrating to Python 3.
The current versions of Python are 2.6 and 3.1, with both Python 2.7 and 3.2 releases imminent.
Even without Python 3 a number of powerful new language features have been added in recent releases. From decorators to generators (and in particular the expansion of generators to provide coroutine functionality in Python 2.5) to conditional expressions, the with statement, abstract base classes and the introduction of new libraries like multiprocessing.
All of this means that the Python community and developers need a chance to adapt to the new features. This is the reason for the Python Language Moratorium (PEP 3003) that mandates no new language features for the next two years. For the developers this means a chance to focus on the standard library rather than adding new language features.
Why did you choose Python?
I kind of got into Python by accident. I had done some programming, including assembly language on the Amiga, up to my time at university but I didn't really use computers for a while after that. A bunch of friends and I were playing a "Play By Email" strategy game called Atlantis and wanted to write a program to automate turns.
Our one constraint on programming language was that there be an implementation for the PocketPC, which was the only computer I had at home at the time. We had more or less decided on Squeak (which would have been a fine choice), but at the last minute someone suggested Python and we went with it.
I really enjoyed programming, and in particular Python, and before long had given up the game but got seriously hooked on Python.
Why should people choose Python?
As it happened Python was a fortuitous choice for me. Python is easy to learn for people new to the language, but is a fully fledged programming language suitable for large application development or small tasks like scripting.
Python is particularly flexible because of the range of programming paradigms and styles it supports. The interactive interpreter (the REPL) makes experimentation easy and fun, and many people do serious work at the interpreter.
New programmers can start with simple scripts, not even needing to write functions if they aren't necessary, moving up through procedural programming to full blown object oriented programming, functional programming and even metaprogramming. With first class namespaces, types and functions all these different programming paradigms are available to the Python programmer.
Python has a rich standard library (it comes "batteries included") that makes it ready to use for a wide range of tasks straight out of the box. There is also a thriving community with a healthy ecosystem of third party libraries and extensions.
The Python language and community extol readability and clarity as the highest virtues in programming. The indentation based block structure, which some people hate but which most people come to appreciate very quickly, is just one way that Python encourages an easy to read programming style.
Comparing with other programming languages, what advantages and disadvantages does Python have?
Python is a highly expressive, dynamically typed language. It is much more productive to use Python than a lower level systems programming language. As one example the codebase of Mercurial (a distributed version control system written in Python) is about 1/10th the size of the git codebase (with comparable features but written in C, Perl and shell scripts). Even when compared to statically typed languages like Java and C#, Python's type system and language features mean that development will typically be faster and result in a smaller code base.
The trade-off is that dynamically typed languages tend to be slower than statically typed languages. On the other hand if you complete the initial implementation faster then you have more time to work on performance optimisation. This really has been my experience - there is much more to be gained in optimising your algorithms than there is from using a different language.
Those who are used to statically typed languages may mourn the passing of type safety in dynamically typed languages, but really type safety is an extremely thin layer of safety. Just because a program compiles doesn't tell you anything about the correctness of the application logic for example. (Otherwise there would never be any bugs in applications written in statically typed languages.) The only way to be serious about the correctness of code is extensive testing (personally I'm an advocate of Test Driven Development, whichever language you are developing in). It just so happens that dynamically typed languages are particularly easy to test because you can create and modify types at runtime and don't have to fight against the compiler. There are many powerful testing libraries and tools available for Python.
How do you comment of Python's rate of usage by people?
Python is very widely used, and has risen in popularity a great deal in the last few years. I think in part the growth of Python recently can be attributed to the publicity around Ruby and Ruby on Rails. Suddenly dynamic languages have become not just acceptable but actually trendy and exciting.
However Python use has been growing in many different areas, not just web development.
Since I first started using Python in 2002 it has become virtually the de-facto administration language for Linux. Pardus is one example of this, Ubuntu is another, where both distributions do everything they can with Python. The Gentoo packaging system, portage, is written in Python and there are plenty of other examples.
Python gets used for science and research, both because it is easy for scientists (who are not primarily programmers) to learn and use and also because of powerful libraries (often wrapping libraries written in C, C++ or Fortran) like Numpy, Scipy, BioPython, matplotlib, NLTK (Natural Language Toolkit) and so on.
Python is also the standard embedded in both the GIS (Geographic Information Systems) and CGI (Computer Generated Imagery) worlds. Large animation houses like Pixar, Imageworks and Industrial Light & Magic all use Python. Applications like Blender and Maya embed Python for scripting.
Python is used for hardware automation, testing systems, application development, web development and just about anything else you can think of. Python is a great general purpose programming language.
Have you ever think to develop an official compiler to create binary files? Or does Python have already?
Well, Python is a highly dynamic language so needs the runtime. It is already a bytecode compiled language but is generally described as an interpreted language because the runtime interprets the bytecode.
Other bytecode compiled languages, like Java and C#, are usually described as compiled languages because they have a Just-In-Time compiler (JIT) that generates and executes native code instead of interpreting bytecode.
There are a few projects to bring a JIT to Python. The most ambitious one is PyPy, an interpreter compiler toolchain that can compile dynamic language interpreters from source code in a static-subset of Python called RPython (Restricted Python). The project also includes a full implementation of the Python language in RPython (plus a smalltalk and prologic implementations). It has native, IL (Intermediate Language - for execution on the .NET and Mono implementations of the Common Language Infrastructure specification by Microsoft) and JVM (Java Virtual Machine) backends - so that a single implementation of a language can be used on all three platforms.
The exciting, and slightly crazy, part of the project is that it also includes a JIT compiler. The PyPy toolchain generates a tracing JIT for language interpreters it compiles. For a long while this was hypothetical and experimental but after several years work there are now lots of benchmarks that run faster on PyPy than they do on CPython (CPython is the standard implementation of Python written in C).
This interview was written before the announcement that Unladen Swallow is aiming to merge with Python 3.
Another project is Unladen-Swallow, sponsored by Google who use Python a lot and have a commercial interest in seeing it run faster. Unladen Swallow uses the LLVM (Low Level Virtual Machine) project to add a JIT to Python. The project includes at least two Python core developers and they see it as a branch rather than a fork; their goal is to merge the performance enhancements back into core Python. The latest releases of Unladen Swallow do show speed improvements over CPython. LLVM is written in C++, and is not brilliantly tested on Windows, both of which could be 'issues' for merging back into Python - so we'll see...
There is also Psyco, which is a specializing compiler that can be bolted-on to CPython. For numeric code in particular it offers massive speed improvements. That project stagnated for a long time, just about maintained but not being developed, but has recently seen a lot of improvements thanks to the work of Christian Tismer.
There are other approaches to creating binary distributions of applications. These typically involve bundling the Python runtime and all the libraries you use together. On Windows a tool called py2exe will do this for you and on the Mac there is py2app. For Linux there is a tool called cx_Freeze and at least one other whose name escapes me at the moment.
Another approach to creating binary applications written in Python is to use a hybrid language like Cython, Pyrex or Shedskin. Shedskin is a static subset of Python that compiles to C++. If you can live without some of the dynamic features of Python you get a compatible syntax, a big performance boost, and compiled binaries. Cython and Pyrex are more often used for writing Python extensions. They are a combination of the Python language with C. You can use Python syntax and semantics mixed and matched with C where you need the speed.
What advantages Python 3 have comparing with Python 2?
The big advantage is that strings and bytes are fully disentangled. In Python 2 we have two string types, the bytestring and Unicode strings. Although it is possible to handle text 'properly' with Python 2 you have to be strict about decoding and encoding at the application boundaries and it is much easier to do-the-wrong-thing and treat text as binary data. If you mix the approaches in Python 2 (sometimes using Unicode and sometimes treating text as binary data) then you're in for a world of hurt (encoding errors in odd places). Python 3 cleans all this up and all strings are Unicode. There is a new bytes type for when you are dealing with binary data. There is a new IO layer in Python 3 to support this.
This alone makes it worth switching to Python 3 in my opinion, and this change couldn't be made without breaking backwards compatibility as it is such a fundamental change.
There are lots of other language changes, there is now only one integer type (unification of the int and long types), comparisons and sorting can only be done with compatible types (the arbitrary ordering of different types in Python 2 didn't really make sense), lots of built-in methods return iterators instead of lists and so on.
The new nonlocal statement is one of my favourite additions in Python 3. It adds true lexical scoping to Python (for changing variables in outer scopes - Python 2 has had lexical scoping for accessing values for a long time). There are also interesting features like function annotations that allow for attaching extra information to function / method signatures. This could be used for enforcing type contracts for example. Programmers are only just beginning to explore some of the things Python 3 makes possible.
The main advantage Python 3 has is that it is the future. Python 2 won't be around for ever and new features and libraries will be added to Python 3 that are never available in Python 2.
It's known that Python 3 doesn't support some of Python 2 codes? Is it really needed?
Unfortunately yes. Some of the changes, particularly the string / bytes changes, just couldn't be introduced in a backwards compatible way. Honestly if it had been possible it would have been done.
What important changes wait for us in future versions of Pyton 2 and Python 3?
One of my favourite changes are all the improvements to the unittest library that come in 2.7 / 3.2.
In Python 2.7 we are trying to restrict new features to ones that are already present in Python 3. This will make the transition between Python 2 and Python 3 easier. In Python 2.7 this means, for example, the addition of the ordered dictionary to the standard library that was added in Python 3.1.
Because of the language moratorium there are very few language changes coming in Python 3.2. There are plenty of other improvements though, including a big improvement to the way floating point numbers are represented (by Mark Dickinson), substantial performance improvements to the Global Interpreter Lock (by Antoine Pitrou), the with statement works with multiple context managers, many bug fixes and so on.
For future development I'd like to see a focus on improving the standard library. In particular although the whole standard library has been converted to work with Python 3 (at least to the point where all the tests pass), a lot of code that previously worked with strings may need careful auditing to see whether it would be better off working with bytes.
When is the end of life time of Python 2?
It looks likely that Python 2.7 will be the last major version of Python. This isn't set in stone, but maintaining the two branches of Python in parallel is a strain on the developers and most of them would rather just be working on Python 3.
(It isn't just two branches - we have Python 2.6 being maintained and Python 2.7 being developed, plus Python 3.1 being maintained and 3.2 being developed. That means bug fixes currently have to be pushed to four branches.)
Even if Python 2.7 is the last major release it will be maintained for several years to come, but the end of life is in sight.
Which GUI (Qt, GTK, etc.) do you prefer while programming with Python?
I've used both Tkinter (bundled as part of the Python standard library) and wxPython in the past. Qt has a very good reputation but the license for using PyQt on Windows was until recently only available under the GPL license or a commercial license. That made it a difficult choice for cross-platform development.
wxPython is ok. It is greatly improved using it from Python with a wrapper library like Dabo that gives it a more Python-like feel.
To be honest my recent GUI development work has all been with Windows Forms for Windows, using IronPython. Surprisingly enough this has a reasonably good API and looks very good on Windows. The Mono implementation of Windows Forms is extensive, but it looks pretty bad by default on the Mac (my main development machine is a Mac). I've seen some very good looking Windows Forms applications run on the Mac with Mono, so I know it's possible I just never learned the right magic tricks.
If the cross-platform license situation improves with PyQt then I would be tempted to use that for future projects. It looks like this is happening in the form of the PySide project, sponsored by Nokia who own the Qt library.
What about Tkinter? Is there any plan about developing Tkinter?
Tkinter looks pretty quirky! It also feels quirky to develop for, but if you just need something simple it is nice to use something that is already in the standard library.
Tkinter is based on Tk, part of the Tcl language. Tk itself has moved on a lot and in Tcl 8.5(?) a fancy new user-interface layer called Ttk became the standard. This is much better looking and has been incorporated into Tkinter in Python 2.7 and 3.2 as the Ttk module.
I've seen screenshots of some very good looking applications written with Ttk, but I have no idea how much magic you have to use to get attractive, native looking UIs or whether you get them for 'free'.
What do you think about Free Software?
I'm a huge fan of free software and very committed to the open source movement. I'm not an enemy of commercial software though. I think it is a good thing that programmers are able to earn a living selling programs. I don't see commercial software and free software as enemies, they can be very good for each other. The development of commercial software, and the money it brings in, sponsors a lot of open source development.
If you use any GNU/Linux distribution, could you say what it is?
I've used Linux on servers regularly. Most servers I've worked with have tended to be Debian based.
Have you ever heard of Pardus GNU/Linux?
I've certainly heard of Pardus, but mainly because of your use and support of Python.
It's known that Python's license is compatible with GPL. Why do you need that?
Well... personally that doesn't make much difference to me. I like, and tend to use for my own projects, the BSD license as they are "more free" than the GPL. On the other hand it would be a major downer if the Python license was incompatible with the GPL and GPL projects couldn't use Python. From that perspective it is important that the Python license is compatible with the GPL.
What's the meaning of developing open source implementation of the Python programming language which is tightly integrated with the .NET Framework according to you?
I see IronPython as a great way of getting developers currently using the Microsoft .NET ecosystem into using free software. IronPython, despite being developed by Microsoft, is fully free software. It's a small step from using IronPython to using Python.
Opposing closed code softwares, free/open source alternatives are written. Wouldn't it be good if an open-source or free alternative of .NET is developed instead of developing IronPython?
There is a very good free implementation of .NET called Mono - and IronPython runs on that too. IronPython is developed by Microsoft and so in no way does the development of IronPython take away from work on Mono.
What's your favorite free software written in Python?
Ha, actually quite a difficult question. Not many of the applications I use regularly are written in Python - except for a couple that aren't free software (the Wing IDE and the Resolver One spreadsheet application). Django is my current favourite free software library written in Python. Developing web applications with Django is great fun.
What do you want to say our readers?
Thank you for the opportunity to speak to you, and thank you for reading what I have to say.
I've recently become involved in a Python related podcast called "A Little Bit of Python". Definitely worth listening to if you're interested in Python.
A Little Bit of Python Episode 3
A Little Bit of Python is an occasional podcast on Python related topics with myself, Brett Cannon, Jesse Noller, Steve Holden and Andrew Kuchling. We still don't have our own website although that is due to land any day now. Meanwhile episode 3 has just gone live. The topics covered include the Python transition to using Mercurial, the release of the first alphas of Python 2.7 and the furore caused by comments on the Python Package Index.
General links for the podcast feeds and a webpage with an embedded flash player:
- A Little Bit of Python mp3 rss feed
- A Little Bit of Python m4a rss feed
- Temporary webpage home for A Little Bit of Python
If you have feedback, insults or suggestions for new topics you can email us on: email@example.com.
We don't yet have the podcast listed on iTunes; we'll set that up as soon as we have a permanent home for the podcast.
This work is licensed under a Creative Commons Attribution-Share Alike 2.0 License.