Python Programming, news on the Voidspace Python Projects and all things techie.
Type Safety and Security
My father is a consultant for safety related computer systems, typically in industrial situations like power-stations, manufacturing plants and gas and oil rigs.
One of the mailing lists he subscribes to is an academic one called 'Safety Critical', and he sent me a pointer to the following thread, which he thought might be of interest.
From the Safety Critical Mailing List :
According to a news item from the Institution of Engineering and Technology, a team organised by the SANS Institute analysed 7000 detected security vulnerabilities from 1996 (they say "the 7000" but don't say further how they were identified), and found that 85% of them were caused by three phenomena:
- Failure to check user input
- Allowing buffer overflows (that is, failing to hinder them)
- Handling integer type checks or overflows incorrectly
Working in a strongly typed programming language would have avoided 85% of the security vulnerabilities discovered (according to some unspecified criteria) in 1996.
It is astonishing to me that 47 years after strong typing was invented and recognised, and after the Turing Award has been presented to such proponents as Dijkstra, Hoare, Wirth, Dahl, Nygaard and Naur, professionals not using this technology caused 85% of significant errors in a specific field of computer science. I think it is disgraceful.
The problem is that people perceive that high productivity can be obtained from using flaky, partially defined, languages such as C++ because of their expressiveness.
Much of the commercial software development market now wants to move away from C/C++ and is looking at or using other languages. Ada 2005 has come too late to mend the OO failings of Ada 95, besides the market wants a true OO language now. The battle is between C# and Java. Of these, C# is better from a programmer's perspective (Microsoft had the benefit of seeing the weaknesses in Java), but Java is more portable. So I expect both to be in widespread use for some time. This doesn't really help the critical systems market, which by and large has good reason to avoid languages which require garbage collection.
First of all it isn't immediately obvious how failure to check user input and allowing buffer overflows are type errors (?), or would have been detected / prevented by a strong type system alone (conflating runtime bounds checking with typing?).
Secondly, Python of course is strongly typed (try adding an integer to a string). In our world the real debate is between static and dynamic type systems. I pointed this out to my Dad (along with links to a couple of relevant Bruce Eckel articles) who passed it onto the mailing list. The response was interesting:
The second article (on Dynamic Typing) was very interesting. It's written from a perspective that I share strongly - that the purpose of allowing people to concentrate on a small number of key things close to the problem domain. It is clear from reading his monologue that he is not working on high integrity systems. He talks of maximising productivity for a low threshold of bugs. He also talks about dynamic (run-time) analysis of data-type information using interpreters. Neither of these reflects typical high integrity practices.
So, picking up his attitude, what should we be doing in the high-integrity software development arena? I think the answer is in two spheres:
- finding environments that allow us to concentrate on the essentials when defining (in detail) the software behaviour
- I believe that is best achieved using simulation models acting as specification
- finding environments for implementing the software reliably
- I believe that should include proof of coherence between specification and implementation
This split follows the concept (but not details) of the "new fad of model-driven architecture", which Bruce Eckel doesn't like. However, in my opinion, the safety community has some different characteristics - primarily being less sensitive to productivity, and more to one particular branch of quality - the one related to evidence supporting the safety case.
[ ... ]
In the commercial programming world, developer productivity is (or at least should be) highly important, as is maintainability. The ability to rapidly create real solutions is a competitive advantage. In the safety critical world, where failure can be catastrophic, not to mention illegal and liability inducing, developer productivity is a much more minor concern.
Web testing with Selenium, and Random Python Stuff
On the subject of writing, Andrew Kuchling has written a new piece of Python advocacy:
It includes examples of embedding Python and extending  Python, and also of using Jython and IronPython (hey, I even get a mention).
There is a new Python related market on Inkling:
This is an old one, by Martin Fowler. On how mocks (for testing) are more than just stubs.
We use a lot of mocks when testing, but usually they are little more than stubs of a few lines. For a long time we've thought it ought to be possible to abstract some good patterns out, and perhaps use a mock library.
Perhaps this article will help me to understand how mocks can be more than just stubs. Giles (the boss) prefers us using stubs than the idea of switching to a general purpose mock library. Mock libraries can automatically provide some of the methods and attributes from the classes you are testing. With stubs you only provide the ones you want, so any attempt to use other aspects of the object API will fail (and alert you of behaviour you weren't expecting or testing for).
|||Which we could possibly use to speed up our functional tests at Resolver. Currently our tests do most things functionally where possible. It sounds like it is actually reasonable to switch to doing the well tested things programmatically for the rest of the tests.|
|||The Python C API, SWIG, BOOST and SIP.|
Python and the Google Summer of Code
Another google summer of code is upon us, and again Python (and Python related projects) have done rather well.
Python related organisations that have had projects accepted include:
- PSF (The Python Software Foundation - Python itself)
- MoinMoin (Wiki)
- Zope Foundation (Web framework)
- wxPython (GUI toolkit)
- Plone Foundation (CMS)
- Django (Web framework)
- BBC Research (Kamaelia - network component project)
- Bazaar (Distributed Source Code Control)
There are some doozies in the Python projects. The one that really caught my eye was :
Trial is the unit testing framework of Twisted. It is based on pyunit and it is used extensively within the Twisted project and elsewhere to run test suites. As a project grows bigger, the size of its test suite also grows polynomially and so does the amount of time required to run it. As of now, there does not exist a nice solution to make these runs take less time. Right now, the Twisted test suite, for example, can take more than 5 minutes to run. This may not seem like a lot of time, but if we take into consideration the number of times these runs need to happen, we realize how convenient it would be if this time could be split.
A Distributed Trial Runner will be able to make these tests run across multiple machines simultaneously, greatly reducing the time needed to complete a run. The tool will allow you to specify which machines to run the tests on, split the tasks accordingly, run the tests on each host and compile the results in the most natural manner.
It sounds like exactly what we need for Resolver.
There are also more gems hiding amongst the other projects:
A Python Programming teaching/testing program, mentored by Ubuntu.
An extension to Chandler (the OSAF Personal Information Manager) to allow scheduling events with natural language.
The HLVM is a set of tools for developing interpreters using the LLVM (Low Level Virtual Machine which is used by PyPy). The HLVM project recently merged with the LLVM, and the aim of this project is to develop a Python front end for the HLVM. Yet another Python implementation! It sounds like a big project for one summer, but could be another interesting tool for experimenting with implementations of Python (extending the language and so on).
OLPC (One Laptop Per Child)
As you might expect, there are several Python related projects for OLPC.
Mono (CLR implementation to run ECMA CLI code on Linux)
None of these projects are directly related to Python, but they have had quite a few accepted this year (including one to implement some of the .NET 3.0 WPF Controls).
Improvements in Mono are good news for the cross platform capabilities of IronPython.
Of course there are swathes of interesting non-Python related projects as well, another fine crop for google this year. We'll have to see how many of these seeds actually get 'harvested' come autumn-time.
Rupy, Time and 'The Book'
Wow, two weeks since my last blog entry. That's the longest I've gone without blogging for a long time. This is due to a combination of the house we rent being sold, attending the Rupy Conference and the ongoing writing of 'The Book'.
I had nearly finished chapter five when it grew beyond the acceptable length for a single chapter. Before I could refactor it into two chapters, Rupy was upon me. I returned from Poznan (Poland) yesterday, and with a bit of luck I should have two chapters finished within the next couple of days.
Rupy was fun. It was a conference on Ruby and Python, organised by Poznan university. Due to the very cheap rates for students it was mainly, but not exclusively, attended by students. About two-thirds of the attendees were interested in Python, but despite the slow submission of Ruby talks their ended up being about an equal number of Ruby/Python talks.
Nice Blog Entry from Andrzej about the conference.
Andrzej and I attended, and talked about Developing with IronPython and Windows Forms. Our talk was based on the one we gave at PyCon, but instead of half-an-hour we had an hour and forty minutes. This meant that we were able to talk a lot more about IronPython itself. Andrzej also did some live coding; adding a new feature (drag-and-drop) to our example TabbedImages Application, following a test driven development approach. This worked very well, no major screw ups, and there was also time for me to show the C# integration using the example of the Clipboard code in TabbedImages.
For those who attended, the presentation is online at:
It was nice to have plenty of time to talk (although it was probably a bit too long), and congratulations to all those who sat through to the end.
I attended a couple of Ruby talks. There was one on Ruby tools, which gave me some interesting ideas for testing in Python. I was disappointed that there were no 'introduction to Ruby or Rails' talks. Luckily Jan Szumiec gave a great talk about writing DSLs in Ruby. The technical level was basic enough that I could follow the code and learned a bit about Ruby in the process . Jan's example was cool - the Brownie Recipe DSL. I think the use of domain specific languages is a great way to focus on expressiveness and readability in code. A lot of the principles (creating a DSL and building the framework behind it to support it) are directly applicable to Python and I hope we see more.
There were other good talks, interesting ones on Turbogears  and SciPy for example. One of my favourite was a talk by the Grono.net, the largest social networking site in Poland. The site is built with Django, and they have had some serious scalability issues to solve.
All in all I think the conference was a success (and Poznan is a beautiful city to visit) and the organizers are pretty sure they will do it again next year. Oh, the super-fast internet connection in the university hotel helped, I was getting download speeds of 1 MByte a second...
|||He says, having now attended three programming conferences in his life...|
|||Or was that list indexing, can't remember...|
|||I didn't know there was a way of catching missing constant lookups in Ruby. That is useful for DSLs. You can get close in Python by executing code in a context (dictionary / namespace) and pre-populating with names, but it isn't quite the same.|
|||After the talk I feel confident enough that I could just start hacking away with Turbogears, which is testimony to both the talk and the project. I don't like XML templating languages though (I don't want to learn a templating specific language for God's sake - just let me use Python in the templates if necessary ).|
|||Which is the approach taken by two extremely successful web development environments - PHP and Rails... Maybe Python doesn't want to be that successful though...|
This work is licensed under a Creative Commons Attribution-Share Alike 2.0 License.