Embedding IronPython in IronPython

The PythonEngine

IronPython as a black box ?

 

 

Note

This is part of a series of articles on embedding IronPython:

For a much more in depth introduction to hosting and interacting with dynamic languages from .NET applications see chapters 14 & 15 if IronPython in Action.

Introduction

This article originally came about from a need to capture standard output from code being executed simultaneously in different places. In the process we discovered that you could create, configure and call the heart of IronPython, the ScriptEngine, from inside IronPython!

This is interesting for executing code in an isolated context [1]. You can also configure the Python ScriptEngine in interesting ways, like setting a custom stream to capture the standard output from the engine. Smile

If you are interested in embedding IronPython in a .NET application, then prototyping and experimenting from the interactive interpreter makes this a much more fun experience.

Simple Embedding

The very basic case, performs the following steps:

  • Create a ScriptEngine; a DLR language engine capable of executing Python
  • Create a new execution scope (effectively a module), with names mapping to values.
  • Create a ScriptSource from the code you want to execute
  • Execute the code in the scope
import clr
clr.AddReference('IronPython')
clr.AddReference('Microsoft.Scripting')

from IronPython.Hosting import Python
from Microsoft.Scripting import SourceCodeKind

code = "print string"
values = {'string': 'Hello World'}

engine = Python.CreateEngine()
source = engine.CreateScriptSourceFromString(code, SourceCodeKind.Statements)
mod = engine.CreateScope()

for name, value in values.items():
    setattr(mod, name, value)

source.Execute(mod)

The result is that 'Hello World' is printed to the console.

Because we are running this code inside IronPython itself, the DLR objects we create act like Python objects rather than .NET objects. If we were doing this from a language like C# then we would use the .NET API on the ScriptScope to add names and values

The Hosted sys Module

The convenience class Python that we used to create the engine in the code above has some other useful methods.

  • GetBuiltinModule
  • GetClrModule
  • GetSysModule
  • ImportModule

Most of these take a ScriptEngine and return something useful. The exception is the last one, ImportModule, which takes an engine plus a module name. It imports the module into the engine and returns the ScriptScope that represents the module.

One of the most useful of these methods, that allows us to configure hosted engines, is GetSysModule

>>> import module
>>> hostedSys = Python.GetSysModule(engine)
>>> hostedSys.path.append('c:\\LibraryDirectory')
>>> hostedSys.modules['module'] = module

Adding Assemblies to the Engine

When you run code with ipy.exe it adds references to the mscorlib.dll and System.dll assemblies. This isn't true of engines you create from the hosting API. We can use the Runtime to add references to these assemblies. The easiest way of getting hold of them from within IronPython is to iterate over clr.References.

>>> import clr
>>> runtime = engine.Runtime
>>> for assembly in clr.References:
...     runtime.LoadAssembly(assembly)

Setting the Standard Output

Something else we can do from the runtime is set custom streams as the standard out and standard error streams for the hosted engine. This allows you (for example) to divert output to a UI component for your application.

We need to create a subclass of the abstract class Stream:

from System.IO import Stream
from System.Text import Encoding

class CustomStrem(Stream):

    def __init__(self):
        self.string = ''

    def Write(self, bytes, offset, count):
        # Turn the byte-array back into a string
        self.string += Encoding.UTF8.GetString(bytes, offset, count)

    @property
    def CanRead(self):
        return False

    @property
    def CanSeek(self):
        return False

    @property
    def CanWrite(self):
        return True

    def Flush(self):
        pass

    def Close(self):
        pass

    @property
    def Position(self):
        return 0

This stream stores anything output on it as a string; its string attribute.

To use it, we call methods on the runtime to set a stream instance as the output stream:

>>> clr.AddReference('IronPython')
>>> clr.AddReference('Microsoft.Scripting')
>>> from IronPython.Hosting import Python
>>> from Microsoft.Scripting import SourceCodeKind
>>> from System.Text import Encoding
>>> from customstream import CustomStream

>>> engine = Python.CreateEngine()
>>> runtime = engine.Runtime
>>> stream = CustomStream()
>>> runtime.IO.SetOutput(stream, Encoding.UTF8)

>>> code = "print 'Hello World'"
>>> source = engine.CreateScriptSourceFromString(code, SourceCodeKind.Statements)
>>> mod = engine.CreateScope()
>>> source.Execute(mod)
>>> stream.string
u'\ufeffHello World\r\n'

Notice that as we are using the UTF8 encoding our string starts with the UTF8 Byte Order Mark (BOM).

With IronPython we can create multiple engines and have each of them execute code simultaneously on separate threads. Unlike CPython IronPython has no Global Interpreter Lock (GIL), so you will see much better performance from multi-threaded code. Using multiple interpreters simultaneously like this is also something that is difficult from CPython.


[1]To make it a real sandbox it would need to be a separate AppDomain of course. This is fully supported by the Dynamic Language Runtime, but will have to be the subject of another article.

For buying techie books, science fiction, computer hardware or the latest gadgets: visit The Voidspace Amazon Store.

Hosted by Webfaction

Return to Top

Page rendered with rest2web the Site Builder

Last edited Fri Nov 27 18:32:35 2009.

Counter...