Dynamic Languages on .NET

Dynamic Languages, IronPython & Silverlight

Silverlight Python

Introduction: Talk Aims

IronPython in Action

I'm Michael Foord.

I have been developing with Python for about five years, and have been working with IronPython for two years with Resolver Systems.

I've written several articles and tutorials on Python, plus I maintain a few Open Source projects in Python. I'm currently writing a book, IronPython in Action, for Manning publications.

IronPython is the first of a 'new wave' of dynamic languages for the .NET platform. These languages can not only be used for developing applications, but also run on Silverlight. Dynamic languages extend the range of choices available to developers and provide readymade scripting languages to extend an application. This talk will be an introduction to dynamic languages on .NET, focusing on IronPython.

Resolver One

Resolver One - Spreadsheet development environment

Can you maintain large projects in dynamic languages?

I'd like to answer this question by showing you what Resolver Systems has done with IronPython.

The three founders of Resolver all worked in the London financial services industry. In that business it is very common for people who aren't programmers to need to build business applications. They don't want to have to go the IT department - they need to be able to create applications for very short term opportunities.

Currently they're all using Excel. Excel is a great application, but beyond a certain level of complexity, the traditional spreadsheet metaphor - of cells in a grid with macros off to one side - breaks down.

So the idea for Resolver One was born - a program where the data and formulae in the grid are turned into code (in an interpreted language) and code that the user writes is executed as part of the spreadsheet.

So how did we end up using IronPython and in fact writing the whole application in IronPython?

Late 2005 two developers started work on Resolver. They chose .NET, and Windows Forms for the user interface, as the development platform, a logical choice for a desktop application. And if you're writing a .NET business application, you write it in C# right? That's what the developers assumed.

But having an interpreted language embedded into Resolver is a central part to the way Resolver One works, so they started evaluating scripting language engines available for .NET. At this time IronPython was at version 0.7 I think. What particularly impressed them about IronPython was the quality of the .NET integration and they decided to see how far they could get writing the whole application in IronPython.

That was almost two years ago. Resolver One is now written (almost) entirely in IronPython, there's over 40000 lines of IronPython production code, plus over 120000 lines in the test framework. Resolver One is free for non-commercial use and can be downloaded from the Resolver Systems website.

The .NET Framework

IronPython

IronPython is a port of the popular programming language Python to the .NET framework. The project was started by Jim Hugunin when he wanted to write an article 'Why the .NET Framework is a Bad Platform for Dynamic Languages'. Jim had already implemented Python for the JVM (Jython), so he knew that Virtual Machines intended to run static languages could support dynamic languages, and he wanted to know why .NET had a reputation for being bad.

As it turned out he discovered that the CLR was actually a pretty good VM for dynamic languages and his 'toy' implementation of Python actually ran faster than CPython! He demoed this fledgling version of IronPython to Microsoft, got hired, and the rest is history.

Why is it called IronPython? It Runs On .NET! (A backronym by John Lam, but it's pretty good.)

Microsoft are serious about IronPython and dynamic languages for the .NET framework. Microsoft have built IronPython support into various projects.

IronPython is very well integrated with the .NET framework. Strings in Python are .NET string objects, and they have the Python methods that you would expect. The same is true for the other types.

We can also take .NET assemblies and import classes / objects from them, and use them with no wrapping needed.

IronPython Integration

IronPython demo!

>>> from System import Array
>>> int_array = Array[int]((1, 2, 3, 4, 5))
>>> int_array
System.Int32[](1, 2, 3, 4, 5)

The only real gap is .NET attributes.

Why Python?

Python is a cross-platform, Open Source, object-oriented, high-level programming language. Older than C# (1991), it is widely used for:

Python is a great general purpose language and so gets used for a wide variety of things. It also grew out of the Unix culture, which is reflected in some of its major uses.

Python is used for web development:

Games:

Animation and CGI:

GIS (Geographical Information Services):

Python has become the 'scripting language of choice' for several major GIS applications (and some are starting to use IronPython).

System admin stuff:

Desktop applications - the original BitTorrent was (and is) written in Python.

Python has been used to implement two major distributed source code control systems: Bazaar and Mercurial.

It is now used internally in Microsoft - e.g. Microsoft Knowledge Tools.

Python gets used a lot in London banking (along with C# - so this another area that IronPython is gaining ground).

Python gets used a lot in science, because it is an easy language for scientists to learn. Particularly for genetic research (because of its powerful string handling capabilities) and linguistic analysis (because of an especially powerful toolkit called the Natural Language Toolkit).

It also gets used in industry, for example Seagate automate their hard drive testing with Python.

The Zen of Python

Readability counts...

Why Dynamic Languages?

Why dynamic languages? A relevant question...

Well, it turns out...

It turns out that a significant proportion of language 'infrastructure' (boiler-plate) is there for the sake of the compiler rather than the programmer.

In C# this includes delegates, interfaces, generics and type declarations which are all obsoleted by a dynamic type system.

Does every slide need a title?

Programs that can be written with a static type system are a subset of all possible programs. For some people this is enough.

—Gilad Bracha (one of the JVM architects)

Face it. The history of programming is one of gradual adoption of dynamic mechanisms.

—Patrick Logan

So What is a Dynamic Language?

Like some other computer science terms, there is no clear definition of what a dynamic language is. Typically it refers to languages that don't do compile time type checking but determine types at runtime.

There are lot of interesting consequences from this, but first it is important to draw a distinction between strong typing and static typing. Some people think that because languages like Python and Ruby are dynamically typed they are also weakly typed. In fact these are different axes. Python is strongly typed, every object has a type and you can't perform operations inappropriate to the type.

Conversely languages like C and Perl are weakly typed. Even though C is statically typed (types declared at runtime) it is weakly typed because you can cast everything to void *.

Multiple Programming Paradigms

Python (and dynamic languages in general) support multiple programming paradigms:

Python is an object oriented language - everything is an object. This doesn't mean that you have to use the object oriented style in your own programs. In fact you don't even need to use functions but can simply create 'scripts' that perform one off operations.

This is exemplified in the interactive interpreter, which as well as being very useful for demos can be a serious programming tool. I know of several .NET programmers who don't use IronPython in their production code, but use the interpreter to explore and experiment with new assemblies. I also know a Python programmer (Python and SQL actually) who does much of his work at the interactive interpreter - he likes to suck some data in, transform it, push it back out and then walk away.

First class functions and types, along with closures (lexical scoping) make functional styles of programming possible.

Metaprogramming through metaclasses, code generation, decorators etc.

Functional Programming

def makeAdder(x):
    def adder(y):
        return x + y
    return adder

def add(x, y):
    return x + y

def partial(func, x):
    def inner(y):
        return func(x, y)
    return inner

Protocols Instead of Interfaces

The mapping protocol (duck-typing):

class Something(object):
    _data = {}
    def __getitem__(self, key):
        return self._data[key]
    def __setitem__(self, key, value):
        self._data[key] = value

>>> something['key'] = 'value'
>>> something['key']
'value'

Customising Attribute Access

Say we want to write something.key instead of something['key']:

class Something(object):
    ...
    def __getattr__(self, key):
        if key in self._data:
            return self._data[key]
        raise AttributeError(key)

>>> something['key'] = 'value'
>>> something.key
'value'

Heterogeneous Container Types

The dictionary can be keyed by any hashable object and contain any object:

>>> locations = {} # dictionary literal

>>> locations[(0, 0)] = {'name': 'origin', 'height': 0.5}
>>> locations[(1, 0)] = {'name': 'corner 1', 'height': 0}
>>> locations[(1, 1)] = {'name': 'corner 2', 'height': 1}
>>> locations[(1, 0)]['name']
'corner 1'

A Memoize Decorator

def memoize(func):
    cache = {}
    def inner(*args):
        if args in cache:
            return cache[args]
        result = func(*args)
        cache[args] = result
        return result
    return inner

@memoize
def do_something(arg1, arg2):
    ...

Generators

Same syntax as functions but use yield instead of return:

def walk(directory):
    for entry in os.listdir(directory):
        path = os.path.join(directory, entry)
        if os.path.isfile(path):
            yield entry
        else:
            for entry in walk(path):
                yield entry

>>> for path in walk('/some/dir'):
...

Tools and Performance

It is easier to optimize correct code than correct optimized code.

—Bill Harlan

Dynamic Languages on .NET

The DLR: a dynamic language runtime that can run dynamic languages on .NET and in Silverlight.

This means lots of dynamic languages - IronPython, Ruby, Visual Basic, Javascript.

Dynamic languages are particularly compelling on the web because it is just text. The XAML is text, the HTML is just text and the code is just text. This makes it much easier for the community to share examples. This is how the web was developed, it is fundamentally a text based technology.

With Silverlight, you can still have the rich experience - but it can all remain text, even when deployed.

The dynamic languages can also interoperate with the traditional .NET languages. They can use classes created from C# /VB code without wrappers. To the dynamic languages they are natural objects just as much as ones they have created themselves.

What you get 'out of the box' with Silverlight 1.1 is IronPython and Managed JScript.

Why managed JScript? Managed Javascript is ECMAScript compliant (ECMAScript 3.0) - so code written AJAX style with traditional Javascript can be ported over to run in Silverlight, and run much faster. Really well integrated with the rest of the platform.

The Dynamic Language Runtime

Silverlight 2

Where did he come from?

Silverlight is a sandboxed browser plugin for creating rich client-side web applications. It runs on Windows and Mac OS, supporting Firefox, IE and Safari (with support for more browsers on the way). Nokia N60 support has been announced

There is a Mono version called Moonlight, which runs on Linux (supporting Firefox). This is now officially backed by Microsoft who are making the media codecs, the test suite and the full specification available to the Mono team. Initially it will target Firefox on Linux, but eventually will run on multiple browsers everywhere that Mono runs.

Moonlight hasn't yet got far down the road of 2.0 support.

Allowing dynamic languages on Silverlight extends the range of programming languages for web applications. This is an area where Flash (the major competitor) fails - only allowing Javascript/Actionscript. It also makes it easier to port existing web applications through Managed JScript.

What Does Silverlight Offer

Through the 'core-clr' we get a rich set of programmers APIs.

Through the DLR dynamic languages can run on .NET and in Silverlight.

This means lots of dynamic languages - IronPython, Ruby, Visual Basic, Javascript.

The dynamic languages can also interoperate with the traditional .NET languages. They can use classes created from C# /VB code without wrappers. To the dynamic languages they are natural objects just as much as ones they have created themselves.

The dynamic languages currently available (from Microsoft) for Silverlight are IronPython, IronRuby and managed JScript.

Why managed JScript? Managed Javascript is ECMAScript compliant (ECMAScript 3.0) - so code written AJAX style with traditional Javascript can be ported over to run in Silverlight, and run much faster. Really well integrated with the rest of the platform.

User interface design can be done with XAML, which separates presentation from code (design from development). XAML is an XML markup for creating user interfaces, including hooking up events and defining animations. This is intended to be created by tools like Expression Blend (free beta and trial versions available). There is also a version for Mono called 'Lunar Eclipse' under development.

The IronPython Web IDE

The IronPython Web IDE

To experiment with IronPython we'll use a simple tool I've created - the IronPython Web IDE.

It comes with several pieces of example code, but also lets you explore the Silverlight API and load and save Python code.

It uses the Javscript EditArea syntax highlighting Python editor.

Standard out is diverted, so that print statements are sent to the 'debugging pane' at the bottom of the screen. Tracebacks are also printed there, to aid debugging.

Python Interactive Interpreter

Interpreter in the browser

This will be fantastic for Python tutorials and documentation.

Embedding IronPython 2

Example 1 - trivially easy. My first calculator...

My first calculator

DLR Hosting Code

engine = ScriptRuntime.Create().GetEngine("py");
scope = engine.CreateScope();

ScriptSource source =
    engine.CreateScriptSourceFromString(expression,
                        SourceCodeKind.Expression);
string result =  source.Execute(scope).ToString();

Plus we have to switch true division on, else 1/2 == 0!

This evaluates an expression and returns a result - would work for any available DLR language. No exception handling shown...

Example 2: The Evaluator

Exposing a scripting API

Exposing a Scripting API

scope.SetVariable("x", x);
scope.SetVariable("button", button);

ScriptSource source =
        engine.CreateScriptSourceFromString(code,
            SourceCodeKind.Statements);

source.Execute(scope);

object x = scope.GetVariable<object>("x");
result = x.ToString();

We can publish objects with events that user code can subscribe to, or fetch objects out of the scope afterwards.

You could create a plugin API by executing user code files and requiring them to define a class or object with a specific name.

Unfortunately IronPython classes are not .NET classes (because you can weird things with dynamic classes like add and remove methods or change the base types - plus .NET classes can't be garbage collected). Working with dynamic objects (classes and instances) directly from C# is painful because you have to do it through reflection. CLR support for dynamic objects will be added. The compiler could automatically generate the reflection calls for you when you work with dynamic objects.

Currently you can get round this either by exposing events (IronPython will create the delegates for you when you add functions to event handlers) or by subclassing a .NET class and casting the object as you fetch it out. That way the compiler knows what members are available on your dynamic object.

IronPython in Action

Questions?