Python Programming, news on the Voidspace Python Projects and all things techie.
Encoding json on Silverlight with System.Json
At the backend of last I year I wrote a blog entry on decoding json on Silverlight. Well, the time has finally come and we're now encoding json to post back to our Django application. The code to encoode json is just as simple as the code for decoding. It makes extensive use of clr.Convert(...) as encoding json relies heavily on implicit conversions:
import clr
from System.Collections.Generic import Dictionary
from System.Json import JsonValue, JsonArray, JsonObject
NoneType = type(None)
def encode_string(data):
return encode_object(data).ToString()
def encode_object(value):
if isinstance(value, (int, long, bool, float, str, NoneType)):
# JsonValue has implicit conversion operators for all
# these types
return clr.Convert(value, JsonValue)
elif isinstance(value, (list, tuple, set)):
return JsonArray([encode_object(obj) for obj in value])
elif isinstance(value, dict):
values = Dictionary[str, JsonValue]()
for key, val in value.items():
values[key] = encode_object(val)
return JsonObject(values)
# raise an error if we encounter an unhandled type
raise TypeError(value)
encode_string takes any data-structure of Python primitive / container types and serializes it into JSON. This json seems to be consumed fine by simplejson on the other end. As well as being simple this code is blazingly fast. The actual encoding operation is done in the .ToString() call, which all happens in C#.
Note
We used to use simplejson inside Silverlight for our json handling. This pulls in several standard library modules and importing them had a big impact on our startup time. Switching to using System.Json was a big win for us both in terms of performance and startup time.
When you send the json over the wire then it will be encoded as UTF-8 for the simple reason that this is the only encoding that Silverlight ships with.
This is relevant because the reason we are shipping huge data structures over the wire is in order to export Excel files from our data-grid views. I initially tried to use xlwt inside Silverlight. Unfortunately it relies on either latin-1 or utf-16 encodings and so my efforts were doomed to failure. Fortunately it works grand on the server side, but the encoding restriction does place limits on the Python libraries that you can use in the browser (unless you are prepared to write your own encoding functions of course).
Like this post? Digg it or Del.icio.us it.
Posted by Fuzzyman on 2010-03-10 00:01:49 | |
Categories: Python, Work Tags: django, silverlight, json, ironpython
Fun with Unicode, Latin-1 and a C1 Control Code
Unicode is a rabbit-warren of complexity; almost fractal in nature, the more you learn about it the more complexity you discover. Anyway, all that aside you can have great fun (i.e. pain) with fairly basic situations even if you are trying to do the right thing.
This particular problem was encountered by Stephan Mitt, one of my colleagues at Comsulting. I helped him find the solution, and with a bit of digging (and some help from #python-dev) worked out why it was happening.
We receive data from customers as CSV files that need importing into a web application. The CSV files are received in latin-1 encoding and we decode and then iterate over them to process a line at a time. Unfortunately the data from the customers included some \x85 characters, which were breaking the CSV parsing.
One of the problems with the latin-1 encoding is that it uses all 256 bytes, so it is never possible to detect badly encoded data. Arbitrary binary data will always successfully decode:
>>> data = ''.join(chr(x) for x in range(256))
>>> data.decode('latin-1')
u'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f...'
If you iterate over a standard file object in Python 2 (i.e. one that reads data as bytestrings) then you iterate over it a line at a time. This splits lines on carriage returns (\x0D) and line feeds (\x0A). If you're on Windows then the sequence \x0D\x0A (CRLF) signifies a new line. If you're trying to do-the-right-thing, and decode your data to Unicode before treating it as text, then you might use code a bit like the following to read it:
import codecs
handle = codecs.open(filename, 'r', encoding='latin-1')
for line in handle:
...
This was the cause of our problem. When decoding using latin-1 \x85 is transcoded to u'\x85', which Unicode treats as a line break. So if your source data has \x85 embedded in it, and you are splitting on lines, where the lines break will be different depending on if you are using byte-strings or Unicode strings:
>>> d = 'foo\x85bar'
>>> d.split()
['foo\x85bar']
>>> u = d.decode('latin-1')
>>> u
u'foo\x85bar'
>>> u.split()
[u'foo','bar']
This could still be a pitfall in Python 3, where all strings are Unicode, particularly if you are porting an application from Python 2 to Python 3. Suddenly your data will behave differently when you treat it as Unicode. The answer is to do the split manually, specifying which character to use as a line break.
The problem isn't restricted to \x85. The Unicode spec on newlines shows us why. \x85 is referred to by the acronym NEL, which is a C1 Control Code: NEL Next Line Equivalent to CR+LF. Used to mark end-of-line on some IBM mainframes.
In fact NEL belongs to a general class of characters known as Paragraph Separators (Category B). This category includes the characters \x1C, \x1D, \x1E, \x0D, \x0A and \x85. Splitting on lines will split on any of these characters, which may not be what you expect. It certainly wasn't what we expected.
For us the solution was simple; we just strip out any occurence of \x85 in the binary data before decoding.
Note
Marius Gedminas suggests that the data is probably encoded as Windows 1252 rather than Latin-1. He is probably right.
There are some interesting notes on Unicode line breaks in this Python bug report: What is an ASCII linebreak?.
Like this post? Digg it or Del.icio.us it.
Posted by Fuzzyman on 2010-01-07 12:42:27 | |
Categories: Python, Work, Hacking Tags: Unicode, latin-1, encoding
Decoding json on Silverlight with System.Json (plus implicit conversion operators in IronPython)
As I explained we're writing a Silverlight application that communicates with our Django server and exchanges a lot of json with it.
Unfortunately, due to what is apparently just an oversight, the codecs module is incomplete for IronPython on Silverlight. This means that recent versions of simplejson don't work.
What we've been using is an older version of simplejson, that pulls in the obsolete and huge sre module in as one of its dependencies. On top of bloating up our application and adding several seconds to the startup importing it all the performance is not exactly blazing.
Fortunately part of the Silverlight SDK is System.Json.dll. Using this API from IronPython is pretty simple. The following code defines a loads function (load string - the same API as simplejson) that takes a string and parses it.
import clr
clr.AddReference('System.Json')
from System import Boolean
from System.Json import JsonValue, JsonType
String = clr.GetClrType(str)
def loads(inString):
thing = JsonValue.Parse(inString)
return handle_value(thing)
def handle_value(inValue):
if inValue is None:
return None
elif inValue.JsonType == JsonType.String:
return clr.Convert(inValue, String)
elif inValue.JsonType == JsonType.Boolean:
return Boolean.Parse(str(inValue))
elif inValue.JsonType == JsonType.Number:
return get_number(inValue.ToString())
elif inValue.JsonType == JsonType.Object:
return dict((pair.Key, handle_value(pair.Value)) for pair in inValue)
elif inValue.JsonType == JsonType.Array:
return [handle_value(value) for value in inValue]
# Should be unreachable - but if it happens I want to know about it!
raise TypeError(inValue)
def get_number(inString):
try:
return int(inString)
except ValueError:
return float(inString)
As with my custom json emitter there is a lot it doesn't do. It handles the following types:
- None
- lists
- dictionaries
- strings
- floats and integers
- booleans
If you want it to handle decmials or dates you'll have to add that yourself. How it works is mostly straightforward, but there is one little piece of 'magic' in there. When you call ToString() (or str() they do the same thing) you get the original json back. For numbers and booleans this is fine as we can easily turn them into the objects they represent. For strings this is a nuisance as we get double quoted, escaped strings. The correct way to get the value we want is to use implicit conversion. In C# this looks something like:
string value = jsonArray["key"];
Declaring the result as a string calls the appropriate JsonValue implicit conversion operator for us. IronPython doesn't have implicit conversion operators as we don't declare types and with a dynamic type system you rarely need to cast. Up until version 2.6 it wasn't possible to call implicit operators, but in IronPython 2.6 we gained a new function in the clr module; clr.Convert:
>>> print clr.Convert.__doc__
object Convert(object o, Type toType)
Attempts to convert the provided
object to the specified type. Conversions that
will be attempted include standard
Python conversions as well as .NET implicit
and explicit conversions.
If the conversion cannot be performed a
TypeError will be raised.
The implicit conversion to string is done with the code:
clr.Convert(jsonValue, clr.GetClrType(str))
I cover some of the other goodies new in IronPython 2.6 (like CompileSubclassTypes) in Dark Corners of IronPython.
I haven't yet written the code to do json encoding, but the JsonValue class (and friends) can be used for encoding as well as decoding, so I expect the code will pretty much be the opposite of the snippet shown above...
Like this post? Digg it or Del.icio.us it.
Posted by Fuzzyman on 2009-12-28 22:56:22 | |
Categories: Python, IronPython, Work Tags: silverlight, json
Django: Tear down and re-sync the database
Django includes the useful management command syncdb for creating the database tables and columns used by your application. If you add new tables (model classes) then re-running syncdb will add them for you. Unfortunately if you modify columns of existing tables, or add new columns, then syncdb isn't man enough for the job.
For modifying the schema of production systems migrations are the way to go. I played a bit with South for Django, which is pretty straightforward. For a system still in development, and changing rapidly, migrations are overkill. We have a script for populating the database with test data, which we update as the schema evolves. (In parallel with this we have a script that imports the original data from the legacy application we are replacing - again updating the script as our app is capable of handling more of the original schema.)
For development what we really want to do is to tear down our development database and re-run syncdb. Running syncdb requires manual input, to create a superuser, so preferably we want to disable this so that the whole process can be automated. I found various recipes online to do this, but mostly using an obsolete technique to disable superuser creation.
In the end I used a combination of this recipe to programatically clear the databases (using the sql generated by sqlclear) and this recipe to disable super user creation.
Note
The code also skips clearing the authentication table as we are using Django authentication unmodified. Comment out the line that does this if you aren't using Django authentication or want to clear it anyway.
#!/usr/bin/env python
import os
import sys
import StringIO
import settings
from django.core.management import setup_environ, call_command
setup_environ(settings)
from django.db import connection
from django.db.models import get_apps, signals
app_labels = [app.__name__.split('.')[-2] for app in get_apps()]
# Skip clearing the users table
app_labels.remove('auth')
sys.stdout = buffer = StringIO.StringIO()
call_command('sqlclear', *app_labels)
sys.stdout = sys.__stdout__
queries = buffer.getvalue().split(';')[1:-2]
cursor = connection.cursor()
for query in queries:
cursor.execute(query.strip())
from django.db.models import signals
from django.contrib.auth.management import create_superuser
from django.contrib.auth import models as auth_app
# Prevent interactive question about wanting a superuser created.
signals.post_syncdb.disconnect(
create_superuser,
sender=auth_app,
dispatch_uid = "django.contrib.auth.management.create_superuser")
call_command('syncdb')
It wasn't all plain sailing. We're using MySQL (God help us) and our development machines are all running Mac OS X. On Mac OS X MySQL identifiers, including table names, are case insensitive. Whilst I would object strongly to a case sensitive programming language this actually makes working at the sql console slightly less annoying so it isn't a problem in itself.
We define our data model using standard Django model classes:
from django.db import models
class NewTableName(models.Model):
NewColumnName = models.CharField(max_length=255, db_column="OriginalSpaltennamen")
class Meta:
db_table = 'UrsprunglichenTabellennamen'
The Meta.db_table specifies the table name that will actually be used in the database. We use the original table and column names where possible as the end users will have to modify some existing tools to work with the new system and this minimizes the changes. As you can see both the original table and new table names are mixed case.
For some reason, which I never got to the bottom of, where the model classes have foreign key relationships syncdb will create these tables with all lowercase names. This could be Django, MySQL or the Python connector to MySQL (or any combination of these) and I never worked out why.
Unfortunately sqlclear will only generate sql to drop tables where the casing specified in the model exactly matches the casing in the database. I worked round it by changing all our Meta.db_table entries to be all lowercase. Not what you would call ideal but acceptable.
Now everytime we update our database schema we can simply run this script. It drops all existing tables and then re-creates them with all the changes.
Note
Carl Meyer suggests using call_command('syncdb', interactive=False) instead of the signals.post_syncdb.disconnect code. It's certainly shorter but I haven't tried it yet.
In the comments Stavros Korokithakis points out that the reset admin command will reset individual apps and regenerate them. If you have several apps in a project this script is still simpler, but if you only need to reset one then you might as well just use ./manage.py reset <appname>. It takes the --no-input switch if you want to supress the user prompts.
Like this post? Digg it or Del.icio.us it.
Posted by Fuzzyman on 2009-12-27 00:06:13 | |
Categories: Python, Work, Tools Tags: django, sql, mysql, databases
A Custom json Emitter for Django
I blogged previously about the limited built-in support for JSON in Django (Python web framework), and how I got around it by first using the wadostuff json encoder and then modifying the one built in to django-piston.
Well, it turned out that json was still a bottleneck in our application. A little bit of background first. I'm working with Comsulting.de on a business application using Django on the server and Silverlight, programmed with IronPython, running in the browser. As the Silverlight application is delivered as a 'xap' file, the application and Django communicate via JSON. This means that we use the Django ORM, URL routing and views, but not the Django templating system.
As I can't program without tests we're testing in the browser with unittest, and to mitigate against slow application startup we do parallel imports in the background (multithreaded using a modified version of ipy-parallel-import) whilst the user is presented with the login screen. If the user logs in before imports have completed they get a progress bar until it is done. The basic framework for all of this is pretty simple and I'll turn it into an article or two when I have time.
The application itself is almost the archetypal business application. The main view is a grid displaying information on about 900 different companies. Users can drill down into details on any company, generate reports, amend details, add contacts and so on. As the main view fetches information on 900 companies, our initial json payload for that view was 2.2 megabytes of json! Unsurprisingly encoding, sneding and decoding that amount of information takes some time. Too much time.
There are various ways we improved performance in our main view; caching the generated json is one way and enabling the gzip middleware was another.
Note
Caching would possibly be a good subject for another blog entry. I ended up using a crude, custom in-database caching mechanism. The existing systems seem to use time based cache invalidation whereas I want unlimited time lifetime but precise control of when to invalidate cache entries programatically. At some point we will probably need to investigate 'proper' caching machinery but our current system is sufficient at the moment.
By far the biggest improvement came from implementing a custom json emitter that only sends the information that we actually need. The default Django json serializer and the modified emitter based on django-piston both send a huge amount of redundant information. With a custom emitter that only sends specified information (yet can work with basic Python data-types and Django model objects) we were able to reduce this initial payload from 2.2 megabytes down to around 500 kilobytes.
The encoding and decoding are both much faster (and fast enough in particular) now. The code for our custom emitter is shown below. No it doesn't handle everything (dates and Decimal for example - and nor would it cope with recursive references), but that is the point. It is simple, it only knows about what we need it to know about and it only sends what we ask for. Just as important it is easy to extend as our requirements grow or change. The work is done by the construct method, whose job it is to turn the object tree we pass in into primitive objects only that simplejson can serialize for us.
from django.db.models import Model
from django.db.models.manager import Manager
from django.db.models.query import QuerySet
from django.core.serializers.json import DateTimeAwareJSONEncoder
from django.utils import simplejson
from django.utils.encoding import smart_unicode
NoneType = type(None)
MISSING = object()
class Emitter(object):
def __init__(self, fields):
self.fields = set(fields)
def render(self, data):
out = self.construct(data)
return simplejson.dumps(out, cls=DateTimeAwareJSONEncoder, ensure_ascii=False)
def construct(self, data):
fields = self.fields
def _any(thing):
if isinstance(thing, (set, tuple, list, QuerySet)):
return [_any(t) for t in thing]
elif isinstance(thing, Model):
# Handle django model objects
return _model(thing)
elif isinstance(thing, Manager):
# For many-to-many relationships
return [_any(t) for t in thing.values()]
elif isinstance(thing, dict):
return dict((_any(key), _any(value)) for key, value in
thing.iteritems() if key in fields)
elif isinstance(thing, (basestring, int, float, long, NoneType)):
# primitive types
return smart_unicode(thing, strings_only=True)
# ha!! (useful for debugging)
raise TypeError('Asked to handle unknown type: %r, %s' % (thing, type(thing)))
def _model(thing):
ret = {}
for member in fields:
attr = getattr(thing, member, MISSING)
if attr is MISSING:
continue
ret[member] = _any(attr)
return ret
return _any(data)
Note
We use select_related in our Django queries which follows foreign key relationships automatically and improved performance by ensuring that the emitter itself rarely results in kicking off new queries.
Using the emitter is simple, we simply have to specify what information we want in the response:
@login_required
def some_view(request):
...
fields = ('FirstName', 'FamilyName', 'eMail', 'PhoneNumber')
emitter = Emitter(fields=fields)
json = emitter.render(in_data)
return HttpResponse(json, mimetype="application/json")
The style of the emitter is very much influenced by django-piston, but the code was all written from scratch so any bugs or failings are entirely my own...
Like this post? Digg it or Del.icio.us it.
Resolver One 1.7, BioPython and 3D Graphics
Although I've left Resolver Systems I still follow closely what they're up to. Resolver One, the IronPython powered spreadsheet with the programming model right at its heart, is an innovative and unique product [1] that deserves to flourish. Despite the horrific loss Resolver One seems to still be moving forward at great pace without me.
Major features in this new release, the last one in which I have been involved in the development [2], include:
- Button-click handlers are now executed without blocking the rest of Resolver One. This means that if you accidentally write a handler that never finishes, or just one that takes longer than you want, you can cancel it while it's running and fix the problem.
- Added long-awaited dialog for setting wrapping and aligment for cells.
- External imported modules now shared between documents and RunWorkbook (better performance for RunWorkbook in particular).
- Faster install time.
- Improved responsiveness when dragging tabs and switching between tabs.
- [Shift] while clicking Recalc toolbar button now now forces reload of all imported modules (like Shift-F9 has always done).
- A few minor bugfixes.
Giles Thomas and team have also produced screencasts of a couple of particularly interesting uses of Resolver One:
Using OpenGL and Resolver One for 3D visualization of stock prices:
This one uses Yahoo! Finance to download the close prices over the last two years for every stock that's currently in the Dow Jones index, then charts them in a 3D window which you can pan and zoom using the mouse. Here's a video showing it in action...
If you're interested in OpenGL Giles has a blog exploring WebGL, OpenGL in the browser: Learning WebGL.
Resolver One includes built-in support for Python C extensions like Numpy through the Resolver Systems sponsored open-source project Ironclad. Ironclad overcomes IronPython's inability to use compiled C-extensions. This video demonstrates the use of one such powerful library, BioPython, from within a Resolver One spreadsheet, to compare the molecular shapes of proteins. (3m12s)
| [1] | In fact Resolver One is a finalist in the 2009 IET Innovations Award. |
| [2] | Possibly not true. I've done some work on the port to IronPython 2.6, which has mainly been spearheaded by Glenn Jones and will hopefully be 1.8. That's assuming that the IronPython team get 2.6 final released soon. |
Like this post? Digg it or Del.icio.us it.
Posted by Fuzzyman on 2009-11-24 15:00:56 | |
Categories: IronPython, Work Tags: resolverone, spreadsheets, opengl, biopython, webgl
Django json support
As I mentioned in my last entry I'm now working on a Silverlight application with Django on the backend. This means that we're using Django to serve json to the Silverlight application, so whilst we're using the Django ORM, url routing and authentication we aren't using its templating.
The data model is 'unusual' but makes sense for the app. We've only implemented the first user story, which uses a subset of the data, but you can already start to see the shape of it. Here's a simplified approximation of the data from the point of view of the Django model classes:
from django.db import models
class CompanyType(models.Model):
type = models.CharField(max_length=255)
class Company(models.Model):
name = models.CharField(max_length=255)
company_type = models.ForeignKey(CompanyType)
class Address(models.Model):
street = models.CharField(max_length=255)
city = models.CharField(max_length=255)
postcode = models.CharField(max_length=255)
company = models.ForeignKey(Company)
class Individual(models.Model):
first_name = models.CharField(max_length=255)
last_name = models.CharField(max_length=255)
address = models.ForeignKey(Address)
The reason for this slightly non-intuitive setup is that a company may have several addresses. At every address there can be several contacts.
In our view we have a companies function that needs to return a list of all the companies. If we use the built-in json serializer then for the company_type field it just puts an id number into the json. If we wanted the actual company_type then we would have to make an additional query per company.
Additionally, for this view we want to retrieve all of the addresses associated with a company and every individual associated with each address.
There is a project called wadostuff that includes a replacement serializer. It's very easy to use, just specify the following in settings.py:
SERIALIZATION_MODULES = {
#'json': 'djangoserializers.json'
'json': 'wadofstuff.django.serializers.json'
}
When we import and call the Django json serializer we can now specify relations for the serializer to follow and include in the json:
from django.core import serializers
from project.app.models import Company
from django.http import HttpResponse
def company(request):
companies = Company.objects.all()
json = serializers.serialize(companies, relations=('company_type',))
return HttpResponse(json, mimetype="application/json"))
This doesn't solve the problem of how we include the addresses and individuals information. One option would be to generate three separate lists and include them all in the json and let the client sort them out. The wadostuff serializer does let us specify a set of extra fields (extras). Despite what the documentation says, in practise I had to implement these as methods on the model objects that could only return a string. This means I couldn't use it to return a list of model objects like I wanted.
Maybe I'm missing something obvious, which is entirely likely as I'm new to Django, but it doesn't seem like this use case is that unusual. I'm surprised that Django has no infrastructure at all to support this kind of use case??
After a bit of hunting I discovered the awesome django-piston project. We don't need an XML or YAML API, nor streaming or throttling, but it includes an awesome json serializer that I 'borrowed' and hacked around so that I could use it on its own. My final code for associating each company with the related addresses and individuals looks like this:
from project.app.models import Company
from project.modules.emitter import Emitter
from django.http import HttpResponse
def companies(request):
companies = Company.objects.all()
for company in companies:
addresses = company.address_set.all()
company.addresses = addresses
for address in addresses:
individuals = address.individual_set.all()
address.individuals = individuals
emitter = Emitter(fields=('company', 'company_type', 'address', 'addresses', 'individuals'))
thejson = emitter.render(companies)
return HttpResponse(thejson, mimetype="application/json")
This follows the 'company', 'company_type', and 'address' relations on model objects it serializes and also handles the addresses and individuals fields. What I get back on the Silverlight end is json representing a list of all companies. Each company has an 'addresses' field with a list of all addresses for that company and each address has an 'individuals' field. This is exactly what we need.
Note
In the comments Doug Napoleone suggests using select_related() rather than all() as it should be more efficient given the way we are using all the relations.
Doug also suggests setting 'related_name' in the model fields which would give me nicer names than address_set and individual_set. If I taught the serializer how to handle these names then I could move the loop from my view to the serializer; but the loop would still be there, so no efficiency gain just nicer looking code.
Like this post? Digg it or Del.icio.us it.
Posted by Fuzzyman on 2009-11-16 01:54:10 | |
Categories: Python, Work, Hacking Tags: django, json, database
New Job with Django and IronPython
Big personal news; I've changed jobs. After more than three years working with Resolver Systems I felt it was time to broaden my development experience. I greatly enjoyed working with Resolver Systems and learned an enormous amount; I'm sorry to leave them but I'm sure they'll manage to cope without me.
I'm now freelance, starting with a contract with a web development firm based in Germany called Comsulting. I first came into contact with Comsulting earlier this year through their lead developer Sven Passig. One of their big customers wanted a web application with the front-end written in IronPython and Silverlight. I did some consulting for them on this as it was their first Silverlight application.
You can hear a bit about what they've been up to with IronPython and Silverlight on the Python 411 podcast that Sven recorded with Ron Jeffries:
I've just been onsite in Germany with them for two weeks but will mainly be working from home. Comsulting's biggest customers are within the German and Swiss media industry and they have several applications for tracking and organising advertising in magazines and websites. It turns out that these media companies really like Silverlight...
We're developing web applications for these companies and after working on the tail end of one project for the first week I started a new project with Stepan Mitt, a designer / developer who also contracts for Comsulting and who happens to be a really cool guy. We're using Django on the server (with CPython 2.5 on Linux) and Silverlight on the client side. I'm converting the Comsulting guys to Test Driven Development, but we still need to investigate the best way to functionally test Silverlight applications.
It's great to be working with CPython again, and especially with Django, but also great to be able to use my IronPython experience. Our Silverlight application communicates with Django via json, so we're using the Django ORM and authentication but our views generate json. I'm sure there will be a blog entry or two out of this.
Like this post? Digg it or Del.icio.us it.
Posted by Fuzzyman on 2009-11-14 20:22:38 | |
Categories: Life, IronPython, Python, Work Tags: silverlight, django
Resolver One 1.6.5 Released
It isn't long since we released version 1.6 of Resolver One, but we're trying to increase the frequency of our releases and get features into the hands of our users faster.
Resolver One is a highly programmable spreadsheet for Windows. Resolver One is written in IronPython and is fully programmable in IronPython. Through the integration of Ironclad you can use Python C extensions like Numpy in the spreadsheet grid and user code.
We've pushed out a new release of Resolver One, and as it is a medium-sized release we've called it version 1.6.5.
The new features in this release our
- Load new files and import from Excel in the background and display a progress bar (leaving the UI usable).
- Support for a new VerticalAlignment property (from user code only - user interface integration is on its way!)
- Fixed bug #2303: Code that checked whether there was a new version to update to could make it look like Resolver One was sending usage information against the user's wishes, even though it wasn't.
- Fixed bug #2268: Memory usage problem when saving huge spreadsheets.
- Fixed bug #2291: Memory usage problem with spreadsheets with very large numbers of errors.
(Plus a few minor bugfixes since the 1.6 release.)
We already have more features completed and on the way, so hopefully we'll be able to do a 1.7 release soon...
In his email to the IronPython mailing list announcing the releas Resolver Systems boss Giles Thomas included the following information about Resolver One:
We are proud to announce the release of Resolver One, version 1.6.5. Resolver One is a Windows-based spreadsheet that integrates IronPython deeply into its recalculation loop, making the models you build more reliable and more maintainable.
It's also still (we think) the largest IronPython application in the world, with 56,000 lines of code backed up by 176,000 lines of unit and functional tests.
For versions 1.6 and 1.6.5, we've made it easier for people to share their spreadsheets. A new free "player" version means you can pass your work on to other people, and they can use it without having to buy anything, while a new Resolverlib makes calling your spreadsheets from IronPython programs as easy as calling a function.
You can read more about Resolver One here:
http://www.resolversystems.com/products/resolver-one/We have a 31-day free trial version, so if you would like to take a look, you can download it from our website:
http://www.resolversystems.com/download/If you want to use Resolver One in an Open Source project, we offer free licenses for that:
http://www.resolversystems.com/opensource/
Like this post? Digg it or Del.icio.us it.
Posted by Fuzzyman on 2009-09-18 22:06:11 | |
Categories: Work, IronPython Tags: spreadsheet, resolverone, release
Resolver One 1.6: Free Resolver Player and Resolverlib
Resolver One 1.6 is out! I say this every time but it's a great release with some important new features. Resolver One is a highly programmable spreadsheet written in IronPython with IronPython scripting support. I covered some of the new features in 1.6 in my blog entry on the Resolver One 1.6 Beta.

The main new features include:
- NumPy support is out of beta - import numpy in user code should "just work", including support for numpy arrays in the grid
- Expanded cells and images in the grid (graphs and charts in the grid!)
- You can now configure the fonts used in the formula bar, code box and console, plus the default font used in the grid
- ZedGraph now included in the distribution to make charting easier
- Auto-complete in the console
- Simpler 'fluent-interface' for creating cell ranges (workbook["Sheet1"].Cells.A1.to.D5)
- Hyperlinks in cells, particularly useful for the web server
- Drag to re-order worksheets
The full list of all the new features and bugfixes since 1.5 is in the Changelog.
Actually, when I said the main new features I was kind of lying, there are two big new features that are to do with re-using Resolver One spreadsheets in new situations.
The first of these is the Resolver Player. This is a cut-down version of Resolver One (without the codebox), intended for distributing the applications you create with Resolver One to end users. Resolver Player licenses are available free on request. You can simulate what Resolver One looks like when one of those keys is installed by choosing "Player mode" from the "View" menu.
The second new feature is Resolverlib. This is a library that allows you to use Resolver One spreadsheets programmatically from IronPython without having Resolver One installed. By embedding the IronPython engine you can use the Resolver One calculation engine, including putting your own UI over it or building it into applications, from any .NET language.
Like this post? Digg it or Del.icio.us it.
Posted by Fuzzyman on 2009-08-31 21:15:28 | |
Categories: Work, IronPython Tags: resolverone, release
IronPython Podcast on FLOSS Weekly and EuroPython Special on Linux Outlaws
Two podcasts with interviews from me on IronPython have been posted this weekend!
The first is an interview with Randal Schwartz and Leo Laporte on FLOSS Weekly:
The podcast is a rambling discussion over about 45 minutes. We discuss topics like, the differences between IronPython and CPython, Ironclad for using Python C extensions from IronPython, Silverlight, the Dynamic Language Runtime (DLR) and so on.
The interview was good fun, but despite exchanging emails with them before the recording (which went out with a live videocast - much to my surprise) they seemed to not be clear what my connection with IronPython was! For the record I'm not a core IronPython developer but work for Resolver Systems who have the largest codebase of IronPython in production. I'm also one of the authors of the first book on IronPython (IronPython in Action) and I maintain the IronPython Cookbook Website and IronPython-URLs Blog.
The second podcast is a Linux Outlaws episode called the "EuroPython special":
It features several interviews recorded at this year's EuroPython conference. Regular readers of this blog will probably recognise most of the names of the interviewees:
On this week's special show, Dan and Fab bring you a handful of interviews from Dan's visit to the EuroPython conference: Steve Holden, Laura Creighton, Bruce Eckel and Michael Foord. We also included the interview Moritz Schmidt-Degenhard from TuxTux did with us.
My part in the show is near the end and lasts about ten minutes or so but manages to cover some of the same ground as the FLOSS Weekly interview, with perhaps a bit more about IronPython on Mono.
- 0:05:39 Steve Holden, Chairman of the Python Software Foundation (PSF) talks about the Foundation and what else he does in the open source world
- 0:13:50 Laura Creighton from Open End talks about the EuroPython conference and also PyPy
- 0:36:12 Bruce Eckel talks about writing technical books, writing them collaboratively and making books available for free on the web
- 0:55:18 Michael Foord from Resolver Systems talks about IronPython and what he does with it at his work (including Ironclad) - also check out his book IronPython in Action
There's a fun comment on the part of the interview with Laura Creighton on the episode forum thread:
She was frighteningly enthusiastic about everything she said. Was Dan using a special interviewing technique, or was it something to do with all the wine she collects? No-one can be naturally that vivacious.
Yup that sounds like Laura, and it's all natural...
Like this post? Digg it or Del.icio.us it.
Posted by Fuzzyman on 2009-08-16 21:10:33 | |
Categories: IronPython, Work, Python Tags: podcast, EuroPython, Ironclad, Mono
Archives
Counter...

