The CPython Server
Getting IronPython & CPython to Talk
Introduction
I have an IronPython application that needs the ability to render images, contour charts specifically. None of the free .NET charting libraries cater for this. What I really want to use is matplotlib, an excellent charting library for CPython.
Unfortunately this makes heavy use of Numpy, a numerical extension for Python written in C, which means that it doesn't work with IronPython.
Wouldn't it be nice if I had a way of executing arbitrary code on CPython and getting a response back?
CPythonServer
Well, I'm sure there are better ways of doing this, but Python provides a web server that is brain-dead easy to extend: BaseHTTPServer.
CPythonServer is a simple 'POST only' web server that receives a Python script as the POST data. It executes this code and extracts the names, 'response' and 'filename' from the context the script is executed in.
Warning
Uhm... this server executes arbitrary Python code. Be careful with it.
'response' should be a string, and is sent as the response. The server uses the extension of 'filename' to determine the mime-type to send the response as.
import BaseHTTPServer
import mimetypes
import shutil
import posixpath
import select
import traceback
from cStringIO import StringIO
__version__ = '0.1.0'
class NoResponseError(Exception):
pass
class CPythonServerRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
server_version = "CPythonServer/" + __version__
def do_GET(self):
"""'/quit' is the only GET allowed ."""
path, query = self.get_query_string()
self.verify_path(path)
self.send_error(501, "Invalid request - POST only.")
return None
def do_POST(self):
"""Serve a POST request."""
f = self.send_head()
if f:
shutil.copyfileobj(f, self.wfile)
f.close()
def send_head(self):
"""
This sends the response code and MIME headers.
Return value is either a file object (which has to be copied
to the outputfile by the caller unless the command was HEAD,
and must be closed by the caller under all circumstances), or
None, in which case the caller has nothing further to do.
"""
path, query = self.get_query_string()
if not self.verify_path(path):
self.send_error(404, "File not found")
return None
data = ''
if self.command.lower() == "post":
data = self.get_post_data()
try:
f, fname, length = self.get_response(path, query, data)
except NoResponseError:
self.send_error(501, "The script did not return a valid response")
return None
except Exception, e:
traceback.print_exc()
self.send_error(501, "%s: %s" % (e.__class__.__name__, e))
return None
ctype = self.guess_type(fname)
self.send_response(200)
self.send_header("Content-type", ctype)
if length is not None:
self.send_header("Content-Length", str(length))
self.end_headers()
return f
def verify_path(self, path):
"""
Does the requested path even make sense?
This implementation has an easy way to quit the server; simply
fetch the path '/quit'.
"""
if path == '/quit':
os._exit(0)
return True
def get_query_string(self):
"Fetch the query string."
path = self.path
i = path.rfind('?')
if i >= 0:
path, query = path[:i], path[i+1:]
else:
query = ''
return path, query
def guess_type(self, path):
"""
Guess the type of a file.
Argument is a PATH (a filename).
Return value is a string of the form type/subtype,
usable for a MIME Content-type header.
"""
base, ext = posixpath.splitext(path)
if ext in self.extensions_map:
return self.extensions_map[ext]
ext = ext.lower()
if ext in self.extensions_map:
return self.extensions_map[ext]
else:
return self.extensions_map['']
if not mimetypes.inited:
mimetypes.init() # try to read system mime.types
extensions_map = mimetypes.types_map.copy()
extensions_map.update({
'': 'application/octet-stream', # Default
'.py': 'text/plain',
'.c': 'text/plain',
'.h': 'text/plain',
})
def get_post_data(self):
"Read the post data."
data = ''
length = self.headers.getheader('content-length')
try:
nbytes = int(length)
except (TypeError, ValueError):
nbytes = 0
if nbytes > 0:
data = self.rfile.read(nbytes)
# throw away additional data [see bug #427345]
while select.select([self.rfile._sock], [], [], 0)[0]:
if not self.rfile._sock.recv(1):
break
return data
def get_response(self, path, query, data):
"""
Get a response file object, filename, and length (if known).
Exceptions raised here will result in a 501 being sent.
"""
print 'Path:', path
print 'Query:', query
print 'Data (first 50 bytes):', data[:50]
context = {}
# Error handling is done above this call
exec (data + '\n', context)
if not 'response' in context:
raise NoResponseError("The script did not return a response.")
response = context['response']
responseName = context.get('filename', '')
return StringIO(response), responseName, len(response)
port = 9981
def run(port=port):
ServerAddress = ('', port)
print 'Serving on port:', port
httpd = BaseHTTPServer.HTTPServer(ServerAddress, CPythonServerRequestHandler)
httpd.serve_forever()
if __name__ == '__main__':
run()
Errors in execution, or no response string set in the context, will result in a 501 error. The tracebacks are printed to the console by the server for debugging purposes.
CPythonServer for Generating Charts
Let's put this into practise. To try this example you will need the CPythonServer running (so you need Python 2.4) installed. You will also need:
clr.AddReference('System.Windows.Forms')
clr.AddReference('System.Drawing')
from System.Windows.Forms import Application, Form, PictureBox, DockStyle
from System import DateTime
from System.Net import WebRequest
from System.Drawing import Image
from System.Text import Encoding
URI = 'http://localhost:9981'
data = """
from pylab import *
from PIL import Image
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from StringIO import StringIO
plot([1, 2, 4, 7, 9], [23, 45, 67, 99, 112])
f = gcf()
canvas = FigureCanvas(f)
canvas.draw()
size = canvas.get_renderer().get_canvas_width_height()
buf = canvas.tostring_rgb()
im = Image.fromstring('RGB', size, buf, 'raw', 'RGB', 0, 1)
imdata = StringIO()
im.save(imdata, format='JPEG')
response = imdata.getvalue()
filename = 'test.jpg'
"""
# Make the request
start = DateTime.Now
request = WebRequest.Create(URI)
request.ContentType = "application/x-www-form-urlencoded"
request.Method = "POST"
bytes = Encoding.ASCII.GetBytes(data)
request.ContentLength = bytes.Length
reqStream = request.GetRequestStream()
reqStream.Write(bytes, 0, bytes.Length)
reqStream.Close()
# Handle the response
response = request.GetResponse()
print response.ContentType
image = Image.FromStream(response.GetResponseStream())
print (DateTime.Now - start).TotalMilliseconds
# Display image
f = Form(Text="Picture from Server")
p = PictureBox()
f.Controls.Add(p)
p.Dock = DockStyle.Fill
p.Image = image
Application.Run(f)
What this does is send a POST request to the server. The data send in the request is a Python script that generates an image using matplotlib.
When run, it produces the following:
The first time the script is executed it takes about 500 milliseconds. If we make the request twice, the second response takes less than one hundred milliseconds. The extra time for the first request is for the import statements (the imports are cached in the CPython process, and don't need to be reloaded the second time).
Having got access to the response stream, it inflates a .NET image using Image.FromStream and displays it in a Windows Forms PictureBox.
To make it easier to read, the matplotlib script used to generate the image (the 'data' string from the script above) is:
from PIL import Image
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from StringIO import StringIO
plot([1, 2, 4, 7, 9], [23, 45, 67, 99, 112])
f = gcf()
canvas = FigureCanvas(f)
canvas.draw()
size = canvas.get_renderer().get_canvas_width_height()
buf = canvas.tostring_rgb()
im = Image.fromstring('RGB', size, buf, 'raw', 'RGB', 0, 1)
imdata = StringIO()
im.save(imdata, format='JPEG')
response = imdata.getvalue()
filename = 'image.jpg'
For buying techie books, science fiction, computer hardware or the latest gadgets: visit The Voidspace Amazon Store.
Last edited Fri Nov 27 18:32:35 2009.
Counter...
