The CPython Server

Getting IronPython & CPython to Talk

IronPython and CPython are friends really

Contents

Introduction
CPythonServer
CPythonServer for Generating Charts

Introduction

I have an IronPython application that needs the ability to render images, contour charts specifically. None of the free .NET charting libraries cater for this. What I really want to use is matplotlib, an excellent charting library for CPython.

Unfortunately this makes heavy use of Numpy, a numerical extension for Python written in C, which means that it doesn't work with IronPython.

Wouldn't it be nice if I had a way of executing arbitrary code on CPython and getting a response back?

CPythonServer

Well, I'm sure there are better ways of doing this, but Python provides a web server that is brain-dead easy to extend: BaseHTTPServer.

CPythonServer is a simple 'POST only' web server that receives a Python script as the POST data. It executes this code and extracts the names, 'response' and 'filename' from the context the script is executed in.

Warning

Uhm... this server executes arbitrary Python code. Be careful with it.

'response' should be a string, and is sent as the response. The server uses the extension of 'filename' to determine the mime-type to send the response as.

import os
import BaseHTTPServer
import mimetypes
import shutil
import posixpath
import select
import traceback
from cStringIO import StringIO

__version__ = '0.1.0'

class NoResponseError(Exception):
    pass

class CPythonServerRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):

    server_version = "CPythonServer/" + __version__

    def do_GET(self):
        """'/quit' is the only GET allowed ."""
        path, query = self.get_query_string()
        self.verify_path(path)
        self.send_error(501, "Invalid request - POST only.")
        return None

    def do_POST(self):
        """Serve a POST request."""
        f = self.send_head()
        if f:
            shutil.copyfileobj(f, self.wfile)
            f.close()

    def send_head(self):
        """
        This sends the response code and MIME headers.

        Return value is either a file object (which has to be copied
        to the outputfile by the caller unless the command was HEAD,
        and must be closed by the caller under all circumstances), or
        None, in which case the caller has nothing further to do.
        """
        path, query = self.get_query_string()

        if not self.verify_path(path):
            self.send_error(404, "File not found")
            return None
        data = ''
        if self.command.lower() == "post":
            data = self.get_post_data()

        try:
            f, fname, length = self.get_response(path, query, data)
        except NoResponseError:
            self.send_error(501, "The script did not return a valid response")
            return None
        except Exception, e:
            traceback.print_exc()
            self.send_error(501, "%s: %s" % (e.__class__.__name__, e))
            return None

        ctype = self.guess_type(fname)
        self.send_response(200)
        self.send_header("Content-type", ctype)
        if length is not None:
            self.send_header("Content-Length", str(length))
        self.end_headers()
        return f

    def verify_path(self, path):
        """
        Does the requested path even make sense?

        This implementation has an easy way to quit the server; simply
        fetch the path '/quit'.
        """
        if path == '/quit':
            os._exit(0)
        return True

    def get_query_string(self):
        "Fetch the query string."
        path = self.path

        i = path.rfind('?')
        if i >= 0:
            path, query = path[:i], path[i+1:]
        else:
            query = ''
        return path, query

    def guess_type(self, path):
        """
        Guess the type of a file.

        Argument is a PATH (a filename).

        Return value is a string of the form type/subtype,
        usable for a MIME Content-type header.
        """
        base, ext = posixpath.splitext(path)
        if ext in self.extensions_map:
            return self.extensions_map[ext]
        ext = ext.lower()
        if ext in self.extensions_map:
            return self.extensions_map[ext]
        else:
            return self.extensions_map['']

    if not mimetypes.inited:
        mimetypes.init() # try to read system mime.types
    extensions_map = mimetypes.types_map.copy()
    extensions_map.update({
        '': 'application/octet-stream', # Default
        '.py': 'text/plain',
        '.c': 'text/plain',
        '.h': 'text/plain',
        })

    def get_post_data(self):
        "Read the post data."
        data = ''
        length = self.headers.getheader('content-length')
        try:
            nbytes = int(length)
        except (TypeError, ValueError):
            nbytes = 0
        if nbytes > 0:
            data = self.rfile.read(nbytes)
        # throw away additional data [see bug #427345]
        while select.select([self.rfile._sock], [], [], 0)[0]:
            if not self.rfile._sock.recv(1):
                break
        return data

    def get_response(self, path, query, data):
        """
        Get a response file object, filename, and length (if known).

        Exceptions raised here will result in a 501 being sent.
        """
        print 'Path:', path
        print 'Query:', query
        print 'Data (first 50 bytes):', data[:50]
        context = {}
        # Error handling is done above this call
        exec (data + '\n', context)
        if not 'response' in context:
            raise NoResponseError("The script did not return a response.")
        response = context['response']
        responseName = context.get('filename', '')

        return StringIO(response), responseName, len(response)

port = 9981

def run(port=port):
     ServerAddress = ('', port)
     print 'Serving on port:', port
     httpd = BaseHTTPServer.HTTPServer(ServerAddress, CPythonServerRequestHandler)
     httpd.serve_forever()

if __name__ == '__main__':
    run()

Errors in execution, or no response string set in the context, will result in a 501 error. The tracebacks are printed to the console by the server for debugging purposes.

CPythonServer for Generating Charts

Let's put this into practise. To try this example you will need the CPythonServer running (so you need Python 2.4) installed. You will also need:

import clr
clr.AddReference('System.Windows.Forms')
clr.AddReference('System.Drawing')
from System.Windows.Forms import Application, Form, PictureBox, DockStyle
from System import DateTime
from System.Net import WebRequest
from System.Drawing import Image
from System.Text import Encoding

URI = 'http://localhost:9981'

data = """
from pylab import *
from PIL import Image
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from StringIO import StringIO

plot([1, 2, 4, 7, 9], [23, 45, 67, 99, 112])

f = gcf()
canvas = FigureCanvas(f)
canvas.draw()

size = canvas.get_renderer().get_canvas_width_height()
buf = canvas.tostring_rgb()

im = Image.fromstring('RGB', size, buf, 'raw', 'RGB', 0, 1)
imdata = StringIO()
im.save(imdata, format='JPEG')

response = imdata.getvalue()
filename = 'test.jpg'
"""

# Make the request
start = DateTime.Now
request = WebRequest.Create(URI)
request.ContentType = "application/x-www-form-urlencoded"
request.Method = "POST"

bytes = Encoding.ASCII.GetBytes(data)
request.ContentLength = bytes.Length
reqStream = request.GetRequestStream()
reqStream.Write(bytes, 0, bytes.Length)
reqStream.Close()

# Handle the response
response = request.GetResponse()
print response.ContentType

image = Image.FromStream(response.GetResponseStream())
print (DateTime.Now - start).TotalMilliseconds

# Display image
f = Form(Text="Picture from Server")
p = PictureBox()
f.Controls.Add(p)
p.Dock = DockStyle.Fill
p.Image = image

Application.Run(f)

What this does is send a POST request to the server. The data send in the request is a Python script that generates an image using matplotlib.

When run, it produces the following:

The first time the script is executed it takes about 500 milliseconds. If we make the request twice, the second response takes less than one hundred milliseconds. The extra time for the first request is for the import statements (the imports are cached in the CPython process, and don't need to be reloaded the second time).

Having got access to the response stream, it inflates a .NET image using Image.FromStream and displays it in a Windows Forms PictureBox.

To make it easier to read, the matplotlib script used to generate the image (the 'data' string from the script above) is:

from pylab import gcf, plot
from PIL import Image
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from StringIO import StringIO

plot([1, 2, 4, 7, 9], [23, 45, 67, 99, 112])

f = gcf()
canvas = FigureCanvas(f)
canvas.draw()

size = canvas.get_renderer().get_canvas_width_height()
buf = canvas.tostring_rgb()

im = Image.fromstring('RGB', size, buf, 'raw', 'RGB', 0, 1)
imdata = StringIO()
im.save(imdata, format='JPEG')

response = imdata.getvalue()
filename = 'image.jpg'

For buying techie books, science fiction, computer hardware or the latest gadgets: visit The Voidspace Amazon Store. If you're looking for a new techie job, try the Voidspace Tech Job Board. This is part of the Hidden Network of technology and programming jobs.