The cgiutils Module

CGI Helper Stuff

Author: Michael Foord
Contact: fuzzyman@voidspace.org.uk
Version: 0.3.6
Date: 2009/06/08
License:BSD License [1]
Online Version:cgiutils online
Support:Mailing List

cgiutils Manual

Note

This module is not under active development and is in 'bugfix only' maintenance mode.

Introduction

The cgiutils module contains functions and constants that are useful when writing Python CGIs. CGI is the simplest way of delivering dynamic content across the internet. Python is particularly good at text processing - which is a major component of delivering html, part of most CGIs.

Of particular interest might be the functions for sending emails programmatically (using sendmail or the smtplib), a function for generating a set of links to many pages, and the DNS blacklist lookup.

There's some other interesting stuff in here as well - like functions to do things that every CGI has to cope with. They are most useful for simple scripts, where a full blown framework would be too cumbersome.

For some uesful Python CGI applications see the Voidspace CGI Page.

This module is a part of the pythonutils [2] package. Many of the modules in this package can be seen at the Voidspace Modules Page or the Voidspace Python Recipebook.

Downloading

As well as being included in the pythonutils package, you can download cgiutils directly from :

Constants

cgiutils contains the following constants :

serverline = "Content-Type: text/html"

# A common location of sendmail on servers
SENDMAIL = "/usr/sbin/sendmail"
validchars = 'abcdefghijklmnopqrstuvwxyz0123456789!-_*'
alphanums = 'abcdefghijklmnopqrstuvwxyz0123456789'

Dealing With CGI Forms

These are a set of convenience functions for handling CGI forms (instances of cgi.FieldStorage). They turn your form into a normal Python dictionary.

You can either use getform or getall directly - or use getrequest, which avoids you having to instantiate a cgi.FieldStorage at all.

These functions don't yet support file uploads [3].

getrequest

getrequest(valuelist=None, nolist=False)

Initialise the FieldStorage and return the specified list of values as a dictionary.

If you don't specify a list of values, then all values will be returned.

If you set nolist to True then any parameters supplied as lists will only have their first entry returned.

getform

getform(valuelist, theform, notpresent='', nolist=False)

This function, given a CGI form (cgi.FieldStorage instance), extracts the data from it, based on valuelist passed in. Any non-present values are set to '' - although this can be changed.

It also takes a keyword argument 'nolist'. If this is True list values only return their first value.

Returns a dictionary.

getall

getall(theform, nolist=False)

Passed a form (FieldStorage instance) return all the values.

Also accepts the 'nolist' keyword argument as getform.

Returns a dictionary.

Other Form Functions

A few more functions that can be convenient for dealing with CGI forms.

isblank

isblank(indict)

Passed an indict of values it checks if any of the values are set.

Returns True if every member of the indict is empty.

I use it on a form processed with getform to tell if my CGI has been activated without any values.

formencode

formencode(theform)

A version that turns a cgi form into a single encoded string.

It only handles single and list values, not multipart.

This allows the contents of a form requested to be encoded into a single value as part of another request.

formdecode

formdecode(thestring):

Decode a single string back into a form like dictionary.

Functions for Handling Email

These functions are used to send emails (including html emails) from Python. They are in this module because it is common to require your CGI application to email you (or the user) when an action is performed.

mailme is the high level function which provides a single interface to sending html and/or text emails. Either via sendmail or smtplib.

For creating an html email (optionally with an embedded text version) then use createhtmlmail.

mailme

mailme(to_email, msg, email_subject=None, from_email=None,
        host='localhost', port=25, username=None, password=None,
        html=True, sendmail=None):

This function will send an email using sendmail or smtplib, depending on what parameters you pass it.

If you want to use sendmail to send the email then set sendmail='/path/to/sendmail'. (The SENDMAIL value from Constants often works).

If you aren't using sendmail then you will need to set host and port to the correct values. If your server requires authentication then you'll need to supply the correct username and password.

to_email can be a single email address, or a list of addresses.

mailme assumes you are sending an html email created by createhtmlmail. If this isn't the case then set html=False.

Some servers won't let you send a message without supplying a from_email.

sendmailme

sendmailme(to_email, msg, email_subject=None, from_email=None,
            html=True, sendmail=SENDMAIL):

Quick and dirty, pipe a message to sendmail. Can only work on UNIX type systems with sendmail.

Will need the path to sendmail - defaults to the 'SENDMAIL' constant.

to_email can be a single email address, or a list of addresses.

Assumes you are sending an html email created by createhtmlmail. If this isn't the case then set html=False.

createhtmlmail

createhtmlmail(subject, html, text=None)

Create a mime-message that will render as HTML or text as appropriate. If no text is supplied we use htmllib to guess a text rendering. (so the html needs to be well formed).

Adapted from recipe 13.5 from Python Cookbook 2.

environdata

environdata()

Returns some data about the CGI environment, in a way that can be mailed. (As a single string).

validemail

validemail(email):

A quick function to do a basic email validation.

Returns False or the email address.

Templating and Output

These functions implement functions performed in one way or another by most CGIs. Outputting to stdout and a basic html templating system.

cgiprint

cgiprint(inline='', unbuff=False, line_end='\r\n')

Print to the stdout.

Set unbuff=True to flush the buffer after every write.

It prints the inline you send it, followed by the line_end. By default this is \r\n - which is the standard specified by the RFC for http headers.

ucgiprint

ucgiprint(inline='', unbuff=False, encoding='UTF-8', line_end='\r\n')

A unicode version of cgiprint. It allows you to store everything in your script as unicode and just do your encoding in one place.

Print to the stdout.

Set unbuff=True to flush the buffer after every write.

It prints the inline you send it, followed by the line_end. By default this is \r\n - which is the standard specified by the RFC for http headers.

inline should be a unicode string.

encoding is the encoding used to encode inline to a byte-string. It defaults to UTF-8, set it to None if you pass in inline as a byte string rather than a unicode string.

replace

replace(instring, indict)

This function provides a simple but effective template system for your html pages. Effectively it is a convenient way of doing multiple replaces in a single string.

Takes a string and a dictionary of replacements.

This function goes through the string and replaces every occurrence of every dictionary key with it's value.

indict can also be a list of tuples instead of a dictionary (or anything accepted by the dict function).

As an example you could use it like :

template = '''
<html>
    <head><title>**title**</title></head>
    <body>
        <h1>**title**</h1>
        <h2>**sub title**</h2>

        <p>**body**</p>
   </body>
</html>'''

#
indict = {
        '**title**': 'This is a Title',
        '**sub title**': 'This is a Sub-Title',
        '**body**': 'This is the body text.'
        }
#
# print the headers
cgiprint(serverline)
# followed by the blank line
cgiprint()
#
# put the values into the template
out_page = replace(template, indict)
#
# print that as well
cgiprint(out_page)

Other Functions

These are a set of functions that don't fit together in convenient categories Smile .

error

error(errorval='')

An ultra-simple generic error function.

This function implements a very dumb error page that reports an error. It uses the string you provide as a level three centered header.

makeindexline

makeindexline(url, startpage, total, numonpage=10, pagesonscreen=5)

Make a menu line for a given number of inputs, with a certain number per page. Will look something like :

First  Previous  22 23 24 25 26 27 28 29 30 31 32  Next  Last

Each number or word will be a link to the relevant page.

url should be in the format : '<a href="script.py?start=%s">%s</a>' - it will have the two %s values filled in by the function.

The url will automatically be put between <strong></strong> tags. Your script needs to accepts a parameter start telling it which page to display.

startpage is the page actually being viewed - which won't be a link.

total is the number of total inputs.

numonpage is the number of inputs per page - this tells makeindexline how many pages to divide the total into.

The links shown will be some before startpage and some after. The amount of pages links are shown for is pagesonscreen. (The actual total number shown will be 2 * pagesonscreen + 1).

The indexes generated are a bit like the ones created by google. Unlike google however, next and previous jump you into the middle of the next set of links. i.e. If you are on page 27 next will take you to 33 and previous to 21. (assuming pagesonscreen is 5). This makes it possible to jump more quickly through a lot of links. Also - the current page will always be in the center of the index. (So you never need Next just to get to the next page).

istrue

istrue(value)

Accepts a string as input.

If the string is one of True, On, Yes, or 1 it returns True.

If the string is one of False, Off, No, or 0 it returns False.

istrue is not case sensitive.

Any other input will raise a KeyError.

randomstring

randomstring(length)

Return a random string of length 'length'.

The string is comprised only of numbers and lowercase letters.

blacklisted

blacklisted(ip, DNSBL_HOST='sbl-xbl.spamhaus.org')

Returns True if ip address is a blacklisted IP (i.e. from a spammer).

ip can also be a domain name - this raises socket.gaierror if the ip is a domain name that cannot be resolved.

The DNS blacklist host (DNSBL_HOST) defaults to sbl-xbl.spamhaus.org.

Other ones you could use include :

  • 'dns.rfc-ignorant.org'
  • 'postmaster.rfc-ignorant.org'
  • 'http.dnsbl.sorbs.net'
  • 'misc.dnsbl.sorbs.net'
  • 'spam.dnsbl.sorbs.net'
  • 'bl.spamcop.net'

Note

Another, possibly more effective, way of coping with spam input to web applications is to use the Akismet Web Service.

For this you can use the Python Akismet API Interface.

TODO/ISSUES

The indexes generated by makeindexline use next to jump 10 pages. This is different to what people will expect if they are used to the 'Google' type index lines.

createhtmlmail assumes iso-8859-1 input encoding for the html

email functions to support 'cc' and 'bcc'

Need doctests

CHANGELOG

2009/06/08 Version 0.3.6

Removed a defunct blacklist server from the blacklisted function.

2005/11/26 Version 0.3.5

Add the blacklisted function.

Added __version__.

2005/10/29 Version 0.3.4

Shortened isblank.

2005/09/21 Version 0.3.3

Fixed bug in getall.

Fixed bug in getrequest.

2005/08/27 Version 0.3.2

Large dictionary replaces use a regex approach.

2005/08/20 Version 0.3.1

Improved istrue function.

Added __all__.

Various other code/doc improvements.

2005/04/07 Version 0.3.0

Changed the email functions, this may break things (but it's better this way)

Added createhtmlemail, removed loginmailme

mailme is now a wrapper for sendmailme, mailme, and the old loginmailme

2005/03/20 Version 0.2.0

Added ucgiprint and replace.

2005/02/18 Version 0.1.0

The first numbered version.

Footnotes

[1]Online at http://www.voidspace.org.uk/python/license.shtml
[2]Online at http://www.voidspace.org.uk/python/pythonutils.html
[3]For file upload by CGI see - http://www.voidspace.org.uk/python/cgi.shtml#upload