Dynamic Languages on .NET

IronPython, Silverlight and Everything

Silverlight Python

Introduction: Talk Aims

IronPython in Action

My name's Michael Foord. I've been a Python developer for nearly five years, and have been developing with IronPython for the last eighteen months or so with Resolver Systems in London. I'm also writing a book for Manning Publications called "IronPython in Action".

IronPython is the first of a 'new wave' of dynamic languages for the .NET platform. These languages can not only be used for developing applications, but also run on Silverlight. Dynamic languages extend the range of choices available to developers and provide readymade scripting languages to extend an application. This talk will be an introduction to dynamic languages on .NET, focusing on IronPython.

Resolver: an IronPython Spreadsheet

Resolver the spreadsheet, development platform and number cruncher

Can you maintain large projects with dynamic languages?

I'd like to answer this question by showing you what Resolver Systems has done with IronPython.

The three founders of Resolver all worked in the London financial services industry. In that business it is very common for people who aren't programmers to need to build business applications. They don't want to have to go the IT department - they need to be able to create applications for very short term opportunities.

Currently they're all using Excel. Excel is a great application, but beyond a certain level of complexity, the traditional spreadsheet metaphor - of cells in a grid with macros off to one side - breaks down.

So the idea for Resolver was born - a program where the data and formulae in the grid are turned into code (in an interpreted language) and code that the user writes is executed as part of the spreadsheet.

So how did we end up using IronPython and in fact writing the whole application in IronPython?

Late 2005 two developers started work on Resolver. They chose .NET, and Windows Forms for the user interface, as the development platform, a logical choice for a desktop application. And if you're writing a .NET business application, you write it in C# right? That's what the developers assumed.

But having an interpreted language embedded into Resolver is a central part to the way Resolver works, so they started evaluating scripting language engines available for .NET. At this time IronPython was at version 0.7 I think. What particularly impressed them about IronPython was the quality of the .NET integration and they decided to see how far they could get writing the whole application in IronPython.

That was almost two years ago. Resolver is now written (almost) entirely in IronPython, there's over 30000 lines of IronPython production code, plus over 100000 lines in the test framework. It's live with several customers in London banking, and going to a public beta phase in the next couple of weeks or so.

So, enough talk, onto showing you Resolver.

What is a Dynamic Language?

Like some other computer science terms, there is no clear definition of what a dynamic language is. Typically it refers to languages that don't do compile time type checking but determine types at runtime.

There are lot of interesting consequences from this, but first it is important to draw a distinction between strong typing and static typing. Some people think that because languages like Python and Ruby are dynamically typed they are also weakly typed. In fact these are different axes. Python is strongly typed, every object has a type and you can't perform operations inappropriate to the type.

Conversely languages like C and Perl are weakly typed. Even though C is statically typed (types declared at runtime) it is weakly typed because you can cast everything to void *.

We'll look at some of the consequences of dynamic typing later, but some of the major ones are listed here.

IronPython

IronPython demo!

Microsoft are serious about IronPython and dynamic languages for the .NET framework. Microsoft have built IronPython support into the following projects:

IronPython is very well integrated with the .NET framework. Strings in Python are .NET string objects, and they have the Python methods that you would expect. The same is true for the other types.

We can also take .NET assemblies and import classes / objects from them, and use them with no wrapping needed.

Why the DLR and Python?

IronPython is a port of the popular programming language Python to the .NET framework. The project was started by Jim Hugunin when he wanted to write an article 'Why the .NET Framework is a Bad Platform for Dynamic Languages'. Jim had already implemented Python for the JVM (Jython), so he knew that Virtual Machines intended to run static languages could support dynamic languages, and he wanted to know why .NET had a reputation for being bad.

As it turned out he discovered that the CLR was actually a pretty good VM for dynamic languages and his 'toy' implementation of Python actually ran faster than CPython! He demoed this fledgling version of IronPython to Microsoft, got hired, and the rest is history.

Why is it called IronPython? It Runs On .NET! (A backronym by John Lam, but it's pretty good.)

So why were Microsoft interested in Python and dynmic languages - why did they create the Dynamic Language Runtime, and in particular, why did they include it in Silverlight?

Dynamic languages are becoming more popular, and Microsoft were aware that developers who preferred these languages were migrating away from .NET.

Python is a particularly clean, expressive and concise language and makes an excellent development language.

Additionally, dynamic languages allow you to mix programming paradigms easily (procedural, functional, object oriented plus metaprogramming). Dynamic languages, without the restrictions of static typing, are also a lot easier to test. The current trend towards Test Driven Development is one of the reasons why some developers are choosing dynamic languages.

One of the reasons that Flash is not beloved by developers, is because you are restricted to a single development language. This is a lesson that Microsoft have learned.

The DLR is especially suited to Silverlight because the web is fundamentally a text based system (even HTTP is essentially a text oriented protocol), and dynamic languages remain text even when deployed.

With IronPython Silverlight applications, the whole project (HTML, Javascript, XAML and Python) can be deployed as text. In practise it seems that communities coalesce more easily around text based systems, as it is easier to share snippets and examples and the edit / compile / run cycle is shorter (by missing out the middle phase!).

Another great advantage of the DLR is that it allows a Managed JScript implementation that follows the ECMAScript 3 standard. This makes it much easier to port current web applications (using interpreted Javascript AJAX style) to run on Silverlight. This code can still use the Silverlight APIs and interact with other Silverlight libraries / code. Managed JScript on Silverlight still runs many times faster than interpreted Javascript.

It is interesting to note that the next iteration of Visual Basic (VB 10) will be a dynamic language built on top of the DLR. This enables Microsoft to provide many of the dynamic language features that they felt developers were asking for.

Python is a great general purpose language and so gets used for a wide variety of things. It also grew out of the Unix culture, which is reflected in some of its major uses.

Python is used for web development:

Games:

Animation and CGI:

GIS (Geographical Information Services):

Python has become the 'scripting language of choice' for several major GIS applications (and some are starting to use IronPython).

System admin stuff:

Desktop applications - the original BitTorrent was (and is) written in Python.

Python has been used to implement two major distributed source code control systems: Bazaar and Mercurial.

It is now used internally in Microsoft - e.g. Microsoft Knowledge Tools.

Python gets used a lot in London banking (along with C# - so this another area that IronPython is gaining ground).

Python gets used a lot in science, because it is an easy language for scientists to learn. Particularly for genetic research (because of its powerful string handling capabilities) and linguistic analysis (because of an especially powerful toolkit called the Natural Language Toolkit).

It also gets used in industry, for example Seagate automate their hard drive testing with Python.

Multiple Programming Paradigms

Python (and dynamic languages in general) support multiple programming paradigms:

Python is an object oriented language - everything is an object. This doesn't mean that you have to use the object oriented style in your own programs. In fact you don't even need to use functions but can simply create 'scripts' that perform one off operations.

This is exemplified in the interactive interpreter, which as well as being very useful for demos can be a serious programming tool. I know of several .NET programmers who don't use IronPython in their production code, but use the interpreter to explore and experiment with new assemblies. I also know a Python programmer (Python and SQL actually) who does much of his work at the interactive interpreter - he likes to suck some data in, transform it, push it back out and then walk away.

First class functions and types, along with closures (lexical scoping) make functional styles of programming possible.

Metaprogramming through metaclasses, code generation, decorators etc.

Duck Typing and Protocols

class SomeClass(object):
    def populate(self, something):
        self._data = something.read()

class DataStore(object):
    def __init__(self):
        self._store = {}

    def __getitem__(self, name):
        return self._store[name]

    def __setitem__(self, name, value):
        return self._store[name] = value

Functional

def makeAdder(x):
    def adder(y):
        return x + y
    return adder

def add(x, y):
    return x + y

def partial(func, x):
    def inner(y):
        return func(x, y)
    return inner

Dynamic Languages on .NET

The dynamic language runtime brings multiple dynamic languages to .NET and opens up the possibility of language interoperability between all the .NET languages:

The DLR: a dynamic language runtime that can run dynamic languages on .NET and in Silverlight.

This means lots of dynamic languages - IronPython, Ruby, Visual Basic, Javascript.

Dynamic languages are particularly compelling on the web because it is just text. The XAML is text, the HTML is just text and the code is just text. This makes it much easier for the community to share examples. This is how the web was developed, it is fundamentally a text based technology.

With Silverlight, you can still have the rich experience - but it can all remain text, even when deployed.

The dynamic languages can also interoperate with the traditional .NET languages. They can use classes created from C# /VB code without wrappers. To the dynamic languages they are natural objects just as much as ones they have created themselves.

What you get 'out of the box' with Silverlight 1.1 is IronPython and Managed JScript.

Why managed JScript? Managed Javascript is ECMAScript compliant (ECMAScript 3.0) - so code written AJAX style with traditional Javascript can be ported over to run in Silverlight, and run much faster. Really well integrated with the rest of the platform.

In their Mix 07 talk, John Lam and Jim Hugunin showed a VB example, using LINQ over objects deserialized from JSON fetched from a web-service. The VB implementation isn't public yet, and the LINQ support in IronPython isn't done - but hopefully you'll also see some cool examples today.

The DLR itself consists of three main parts:

  1. Dynamic type system, shared type system. Shared with existing .NET languages.
  2. Language hosting system.
  3. Compiler system (with parser and AST).

IronPython on Silverlight

Silverlight Photo Viewer

The IronPython Web IDE

www.voidspace.org.uk/ironpython/webide/webide.html

The IronPython Web IDE

To experiment with IronPython we'll use a simple tool I've created - the IronPython Web IDE.

This is available online and you can also download it as a project.

It comes with several pieces of example code, but also lets you explore the Silverlight API and load and save Python code.

It uses the Javscript EditArea syntax highlighting Python editor.

Standard out is diverted, so that print statements are sent to the 'debugging pane' at the bottom of the screen. Tracebacks are also printed there, to aid debugging.

The Video Player

Using the MediaElement class from IronPython.

from System.Windows.Controls import MediaElement
from System import TimeSpan, Uri, UriKind

m = MediaElement()
u = Uri('HelloWorld.wmv', UriKind.Relative)
t = TimeSpan(0)
m.Source = u
m.Position = t

Accessing the Browser DOM

Changing HTML elements and setting style attributes.

from System.Windows.Browser import HtmlPage

html = '<strong>Set from IronPython</strong>'
HtmlPage.Document.experimental.innerHTML = html
e = HtmlPage.Document.GetElementByID('experimental')
e.SetStyleAttribute('border', 'solid black 2px')

Open File Dialog

from System.Windows.Controls import (
    OpenFileDialog, DialogResult
)

dialog = OpenFileDialog()
dialog.Title = "Load Python File"
dialog.Filter = "Python files (*.py)|*.py|All files (*.*)|*.*"
if dialog.ShowDialog() == DialogResult.OK:
    print dialog.SelectedFile.OpenText().ReadToEnd()

IsolatedFileStorage

from System.IO.IsolatedStorage import (
    IsolatedStorageFile, IsolatedStorageFileStream)
from System.IO import (
    FileMode, StreamReader, StreamWriter)

s = IsolatedStorageFile.GetUserStoreForApplication()
i = IsolatedStorageFileStream(name,
        FileMode.OpenOrCreate, s)

writer = StreamWriter(i)
writer.Write(data)
writer.Close();i.Close();s.Close()

Silverlight provides 1 megabyte of local storage ('in the browser') per application. You access it through the IsolatedStorageFile class.

Whilst I could create subdirectories in the file store, and save/load files from them, listing files in subdirectories didn't seem to work. Listing files in the root directory worked fine though.

Downloader

Accessing server resources (but no cross-domain calls).

from System import Uri
from System.Windows import Downloader

uri = Uri('http://www.voidspace.org.uk/')
dl = Downloader()

dl.Completed += someFunction
dl.Open('GET', uri)
dl.Send()

You can make synchronous calls with a class called the BrowserHttpWebRequest which has a more sophisticated API for doing POSTs and accessing headers etc.

Button

Subclassing the Control class (and consuming XAML).

class Button(Control):
    def __init__(self):
        root = self.InitializeFromXaml(xaml)
        self.button = root.FindName("button")

        self.button.MouseEnter += self.onMouseEnter
        ...
        self.Click = EventHook()

The XAML isn't shown here, but is in the Web IDE example.

Creating a Silverlight control. This inherits from the Control class provided in System.Windows.Controls. It is initialized from XAML. The events are then hooked up in the constructor. I've used my own Python EventHook class to create the Click event.

Embedding: A Custom Executable

using IronPython.Hosting;
using IronPython.Runtime;

static void Main(string[] rawArgs) {
    List<string> args = new List<string>(rawArgs);
    args.Insert(0, Application.ExecutablePath);

    PythonEngine engine = new PythonEngine();
    engine.Sys.argv = List.Make(args);

    EngineModule engineModule = engine.CreateModule("__main__",  new Dictionary<string, object>(), true);
    engine.DefaultModule = engineModule;

    string path = Path.Combine(Path.GetDirectoryName(Application.ExecutablePath), "main.py");
    engine.ExecuteFile(path);

IronPython was designed from the start to be embeddable in .NET applications.

This is an IronPython 1 example.

IronPython 2 Hosting API

clr.AddReference('IronPython')
from IronPython.Hosting import PythonEngine
clr.AddReference('Microsoft.Scripting')
from Microsoft.Scripting import ScriptDomainManager


TEXT = 'hello world'
mod = ScriptDomainManager.CurrentManager.Host.DefaultModule
mod.SetVariable("text", TEXT)

engine = PythonEngine.CurrentEngine
engine.Execute('print text')

IronPython in Action

IronPython in Action