Python Programming, news on the Voidspace Python Projects and all things techie.
CPython Extensions from IronPython: With Python.NET
Resolver Systems recently announced a new project to get CPython extensions working seamlessly with IronPython. This is needed by customers of Resolver, but is being run as an Open Source project.
Updated to version 0.1.1.
This allows you to pass keyword arguments to CPython functions / methods from IronPython.
- Announcement: Project to get some CPython C extensions running under IronPython
- C Extensions for IronPython Mailing List
One possible approach is to embed the CPython interpreter in an assembly, and access CPython extensions through the hosted interpreter.
I spent a day of Resolver time exploring this possibility and discovered that most of the hard work had already been for us by the Python.NET project.
After less than a days hacking I have a rough proof of concept working. Despite some serious limitations it works! If you have a desperate need to use CPython extensions from IronPython you can probably already build on what I have done. I have already had matplotlib with numpy and Tkinter working from IronPython.
For the details, including code:
Squeak on PyPy
PyPy is still going strong. It still holds out the possibility of faster than CPython execution with a JIT based on psyco, sandboxed execution and fast RPython extensions compiled to C or .NET assemblies.
In a new project, at a sprint in Bern, people are working to implement a Squeak-bytecode interpreter in RPython.
Oh, and in totally unrelated news, my talk proposal for the Developer Developer Developer UK Community Event has been accepted, yay.
Resolver Reviewed by Andrew Dalke
Resolver Systems is gradually extending the private beta of our IronPython spreadsheet, Resolver One. We've had a lot of interest from people working in the bioinformatics field (which was a great surprise to us - we were targeting the financial services market), so one of the people to get an early beta was Andrew Dalke who works in chemical informatics and is a regular blogger about Python.
He has blogged his first impressions of Resolver.
His blog entry was particularly pleasing to me as Andrew has discovered and made good use of my Resolver Hacks site.
Overall (I think) his impressions are good but his blog entry made some good suggestions and asked some interesting questions. Here is a response to some of his questions (summarised):
- The PYTHONPATH isn't set.
When you are editing a Resolver spreadsheet, the directory containing the spreadsheet plus the Resolver library directory is on the path for importing modules from. We have a user story for an interface to add directory / files to the path - but perhaps obeying the PYTHONPATH is a good idea.
- The current directory is wrong / changes.
Resolver works within a single process. That means if you have more than one document open then they share a current directory. Unfortunately using the open file dialog changes the directory as well (not our fault!). There is an easy way round this though. You have access to the full path to the current spreadsheet as the __file__ variable. You can access files relative to this path if you need to.
- I want my own editor!
Allowing users to edit user code sections in an external IDE, with Resolver watching the files and recalculating on changes, would be a great improvement. It's on the list!
In the meantime there is an easy way to achieve this with execfile though. I thought it was a neat idea, so I wrote it up on the Resolverhacks site: So You Want Your IDE Back.
- Resolver can appear to freeze on loading
When you load or import a document, Resolver does its first recalc which populates all the caches. We don't currently have a progress bar whilst this happens, but we've 'spiked' it and it is easy to add.
In the longer run, we will save results as well as formulae in the save file. This is needed so that you can reference external Resolver documents from formulae, but will also mean that we can display values immediately.
- Distributing spreadsheets with dependent code modules means sending several files
A possible solution to this is to make Resolver save files zip archives. This would reduce their size, but we could also allow other files to be stored in the archive and provide an API for loading / saving / importing files from the archive. Then spreadsheets and dependencies can be distributed as single files. We would have to make sure that this still works with some method of using an external editor though.
- Triggering a recalc from asynchronous events
This is one of the areas where Resolver needs to grow. It is already possible with an external process watching the event(s) and using Resolver's database trigger listener feature to trigger a recalc. There is an example on Resolverhacks of watching a data file for changes and triggering a recalc.
- Context menu option to jump from generated code to the corresponding cell
We like this user story! We already had one for the other way round, but this is an even better idea.
- Editing generated code
Because of the way the formula language is defined, formulae can always be turned into Python. Arbitrary Python code cannot always be turned back into a valid formula though. However, we do have an idea for an alternative formula style that would allow you to use arbitrary Python expressions instead of our formula language. It has to be an expression, because it goes on the right hand side of an assignment.
For example, a formula in cell A1 becomes an assignment to the value of cell A1:
workbook['Sheet1'].A1.Value = <Python generated from the formula>
We could do something like allowing this for formulae:
=@(arbitrary Python expression in here)
Then we could make the code generated from formulae editable (only the code to the right of the assignment of course). An edited formula would be put back into the cell using the syntax above.
Other questions and comments:
Plotting, uhm... I mean charting.
This is something that is high on the list for implementation. A decent charting solution with a high level interface will be provided soon. For scientific data, gnuplot is probably more suitable. With the preliminary work I've done on using CPython Extensions with IronPython it should already be possible to use matplotlib with Resolver.
Limits on data in spreadsheets.
There are no limits on the amount of data you can store in a Resolver spreadsheet. You can't use cell references beyond column 'ZZZ' (column 17576) in formulae, but you can use numbers to access the cells.
2D and 3D charts in cells with pop-up windows to manipulate them
Great ideas. Images in cells will come with the charting API. Popping up a custom form from a button linked to spreadsheet data is already possible (as a blocking dialog), you just have to write the UI yourself!
We'd like to develop a testing user interface into Resolver. We think spreadsheets could massively benefit from testing. Possibly a 'test mode' that allows you to provide a test data set and make assertions about the results of formulae.
Version control for code
Andrew's problem with formulae and crosses in cells
A cross inside a cell indicates an error. There should be a tooltip to explain the error (and an error traceback gets sent to the output view - the bottom pane of Resolver). I still couldn't work out what Andrew's problem was without knowing the error message.
Python for Windows Mobile Devices
I no longer have a PDA running Windows CE (coff iPhone) or any of its multipluous variants , but mobile devices seem to be becoming ever more important - and a lot of them are running Windows. For this reason it's great that PythonCE is still being actively developed.
The mailing list is seeing regular reports of people getting Python (including user interfaces) to run on smartphones and many different devices. There are also a range of tools and libraries available for working with PythonCE:
- List of PythonCE Tools
- Remote-console project
- ceserial for working with the serial port on Windows CE - apparently this works for real serial port and the bluetooth serial port profile
There is also the promise of a forthcoming release of a new (and less raw than Venster) GUI for PythonCE: PocketPyGui.
Several of the latest Python 411 Podcasts have covered Python on smartphones and other mobile devices, so if you are interested there is plenty to keep you busy!
(Oh - if you'e underwhelmed with the iPhone hardware, check out this review of the new HTC Phone: the HTC Kaiser. The specs look awesome but the blowaway factor with the iPhone is the user interface not the hardware.)
In case you're wondering, Python runs fine on an unlocked iPhone. I have Python 2.5 with the VT100 terminal. No UI toolkits that I know of and I haven't checked to see if ctypes works (it is now part of the PythonCE distribution).
|||Windows CE, Windows Embedded, Pocket PC and Windows Mobile to name a few.|
There have often been calls for an Ordered Dictionary to be added to the Python standard library. The latest of these was a long discussion on comp.lang.python: An ordered dictionary for the Python library.
The most common requirement is for a dictionary where keys are ordered by insertion order, but people also want keys sorted alphabetically and ordered by most recently used keys. This disagreement, no consensus on 'one size fits all', is the reason why there isn't an implementation in the standard library. Thus the debate continues...
A while ago Nicola Larosa and I created OrderedDict. This is a Python dictionary like object that orders keys by insertion order, written in Pure Python. Operations like iteration and slicing are supported, plus methods like .keys(), .items() and .values() all observing the order.
Anthon van der Neut has created an ordered dictionary in C.
Being implemented in C, this version is nearly as fast as the built in dictionary type! It was even influenced by the 'Foord/Larosa' version - but it doesn't yet support everything that ours does.
MixUK Videos Available
The videos from the Microsoft Mix UK conference are now available. This includes my talk on dynamic languages on .NET.
My talk is on day 2 (14:00 to 15:00 in track 1) and is actually called: IronPython et al: Using Dynamic Languages in .Net and Silverlight.
The talk description is pretty good, but I don't remember writing it!
This unprecedented level of integration is possible because of the new Dynamic Language Runtime (DLR) and the Silverlight browser plugin. Rich internet applications can be created using existing tools combined with the power of Silverlight.
ReStructuredText is a lightweight text markup for writing documentation. It's a fantastic way of generating PDFs, LaTeX or HTML from plain text. There are no shortage of text markups, but ReStructuredText is particularly good. The goal of the creators was to create a WYSIWYG markup, so that documents using ReST are easily readable as text (and as a result the markup syntax is easy to learn).
Recently there has been a rash of tools coming out that support ReST.
Rest in Peace
Rest in Peace is a ReStructured Text editor. It has a browser pane so that you can see the results of your document whilst you edit it. It's still an early version but worked very well whilst I was trying it. There is a new beta just out as well:
It needs Python 2.5 and has the following dependencies:
Which I assume means ReStructured Text to anything. This is a webservice / website that converts ReStructured Text to HTML / PDF, and lets you choose the output style.
Pandoc is a Haskell library for converting between markup formats. It can read and write various markup formats, including ReST. It can even read HTML and turn it back into ReST!
This work is licensed under a Creative Commons Attribution-Share Alike 2.0 License.