Python Properties and C# 3.0 Language Features

New Ways of Doing Properties, LINQ and Other Stuff

Contents

Introduction
Properties in Python and CSharp
Python Properties Inspired by C# Syntax
New Language Features in C# 3.0
- Catchup Language Features
- New and (Possibly) Innovative Features
Implementing LINQ for Python

Introduction

This article is inspired by a talk I went to at TechEd Barcelona about new language features in C# 3.0: perhaps there is something that Python can learn from then!

One of the features is an improvement to the way trivial properties are declared in C#. The C# syntax for declaring properties is cleaner than Python, despite recent changes implemented by Guido.

This article starts by looking at property syntax in Python (old and new) and C#, including a new way that I have implemented inspired by the C# syntax. This uses metaclasses and descriptors (under the hood), so it is pretty fun geek stuff even if you don't use it!

The middle part of the article is a look at some of the other new language features in C# 3, and which of them Python might be able to learn from. It finishes with a look at the major new language feature (LINQ) and whether it is possible to add this to Python.

Properties in Python and CSharp

The current way of declaring properties in Python is a little awkward:

class Class(object):
    def __init__(self):
        self.__attribute = None

    def __getattribute(self):
        return self.__attribute

    def __setattribute(self, value):
        self.__attribute = value

    attribute = property(
        fget = __getattribute,
        fset = __setattribute
    )

Of course if you want a read-only property, you can use the property decorator which is much nicer:

class Class(object):
    def __init__(self):
        self._something = None

    @property
    def something(self):
        return self._something

Guido has just checked in an improvement to the way you create properties. This will appear in Python 2.6 and 3.0, and lets you use decorators to declare the setter and deleter methods as well:

class Something(object):
    def __init__(self):
        self._something = None

    @property
    def something(self):
        return self._something

    @something.setter
    def something(self, value):
        self._something = value

    @something.deleter
    def something(self):
        del self._something

This is a great improvement, but it still isn't as clean as the property syntax in C#:

class Something {

    private Object _something;

    public Object something {
        get { return _something; }

        set { _something = value; }
    }
 }

Excluding the deleter (which C# doesn't allow) (and ignoring whitespace and curly brace lines), C# is five lines and Python nine lines.

Using a metaclass and descriptors, we can get something similar to the C# syntax in Python.

Python Properties Inspired by C# Syntax

Note

I've also implemented another way of automatically declaring properties in Python; by declaring methods that begin with 'get_', 'set_' or 'del_':

Automatic Properties with get, set and del methods

Using a metaclasses and the descriptor protocol, we can get the equivalent of the C# syntax in Python. It isn't many lines shorter, but perhaps more readable than the current solution (and is available today):

class Something(object):
    def __init__(self, x):
        self.x = x

    class something(Property):
        def get(owner):
            return owner.x
        def set(owner, value):
            owner.x = value

s = Something(3)
print s.something

s.something = 4
print s.something

The property is the inner class, with get and set methods. The magic behind the scenes is the Property class and its metaclass Meta:

from types import FunctionType, MethodType

class Meta(type):

    def __new__(meta, classname, bases, classDict):
        for name, entry in classDict.items():
            if isinstance(entry, (FunctionType, MethodType)):
                classDict[name] = staticmethod(entry)
        return type.__new__(meta, classname, bases, classDict)

    def __get__(self, instance, owner):
        return self.get(instance)

    def __set__(self, instance, value):
        return self.set(instance, value)

class Property(object):
    __metaclass__ = Meta

It's a bit magic, but fun. Your properties, implemented as inner classes, must inherit from Property. This has a metaclass, so that all methods get turned into static methods. Because the metaclass implements __get__ and __set__, they are used when the property class is fetched on an instance. This descriptor magic makes sure that getting or setting the property does the right thing - including giving you access to the instance that contains it.

The reason that I started thinking about this is that 'Automatic property declaration' is one of the new features of C# 3.0.

New Language Features in C# 3.0

C# 3.0 is part of the .NET 3.5 and introduces new language features. A lot of the new language features are actually to support the major new feature of C# 3.0, which is LINQ. Some of these features are just C# playing catchup with dynamic languages like Python.

Catchup Language Features

Automatic property declaration
A simple syntax for trivial getter/setter properties. In Python you would just expose an attribute:
```
class Something {

    public string something { get; set;}
 }
```
The reason why you might use a trivial property like this (in C#), rather than just expose a field, is that it allows you to change the code later without changing the public API (and so require recompiling of code that uses your API). In .NET if you compile against an API that accesses a field, then it generates different code than accessing a property. If you change a field to a property then your new assembly isn't a drop in replacement.
Lambda expressions:

c => c.City == "London"

This is a construct very similar to the Python lambda function. Under the hood the compiler constructs an anonymous delegate of type Func. Almost a function.
Extension methods

Adding new methods to classes and classes that implement interfaces. These new methods call static methods on another type. Although adding methods to classes is easy in Python, you can't do it to builtin types. In C# 3.0 you can add a new method to all classes that support enumeration (for example). Being able to add extension methods to the builtin types would bring a feature beloved in Ruby to Python.

With Python 3.0 we will gain new abstract base classes. Would it be acceptable (or more to the point useful) in Python to be able to add methods to all mapping types or all sequence types?
Limited type inferencing

At least now you usually only need to declare the type once! (Although you have to explicitly use the var keyword.) In some cases type is genuinely inferred, like lambda expressions and anonymous types (see below).

New and (Possibly) Innovative Features

There are also some language features that are new and interesting. Some of them wouldn't be out of place in Python, but might be very hard to implement.

Anonymous types

How many times have you created a simple type just to expose a couple of members. A sort of named record type. I believe there is a new named record type going into Python (but not as a builtin), but C# 3.0 actually allows anonymous types:

{Name=something, Value=something_else}

This creates an anonymous type with 'Name' and 'Value' fields (attributes). Because the type isn't named, you can effectively only use these objects within the body of a single method. Elsewhere you can't declare this type anywhere because the specific type is generated by the compiler. Still, the syntax / feature is quite nice and I wouldn't mind seeing it in Python. They are particularly useful for LINQ expressions mentioned below.
Automatic property initialisation
Not sure what I think of the syntax of this one. It is also particularly suited to the .NET framework because it is a very common pattern to have an empty constructor and initialise an instance by setting several properties.

This C# 3.0 syntax:
```
var something = new SomeClass(){
    Name=something,
    Value=something_else
}
```
Is the equivalent of:
```
var something = SomeClass();
something.Name = something
something.Value = something_else
```
In Python I guess you would just use keyword arguments in the constructor (and in fact in IronPython already allows you set .NET properties by passing in keyword arguments to a constructor), so on balance I don't think Python has anything to learn from this.
LINQ - Language integrated queries
This feature actually introduces language syntax for making queries. These queries are against data providers - for example SQL, XML or objects. For simple LINQ over objects this is very similar to list expressions in Python. The nice thing is that data providers are free to optimise the query, for example generating nice SQL from a query.
```
var collection = from c in customers
    where c.City == "London"
    select {c.Name, c.Email}
    orderby c.Name;
```
You can see how the anonymous type is used in the select statement. In Python you would simply use a tuple, but I think an object with named fields is nicer. c.City == "London" is the filter, and is a lambda expression. LINQ over objects is implemented as a foreach loop (exactly like a Python list comprehension except you get things like orderby). If the query was against another data provider, they receive an expression tree that represents the query, and they get to turn to the expression tree into whatever query they want under the hood.

LINQ can be used for much more than simple select statements like this and is a flexible language addition. If list comprehensions could optionally call an API (a protocol method) instead of being always executed directly then partial support for something similar could be added using the list comprehension syntax. This might impact performance though and see the strong caveat below about expression trees.
Expression trees

An expression tree is a representation of an expression as a form of AST. It is through expression trees that data providers are able to understand the lambda expressions used in LINQ and generate an optimised query for their data source.

Using the expression tree API directly is inevitably going to be fairly complex, but in C# allows you to create new data providers.

Implementing LINQ for Python

If Python was to add some LINQ like support within list comprehensions (which LINQ already closely resembles) then it would need to generate an expression tree for the filter part of a list comprehension. The data provider would then need to be able to decompose the expression tree into a query.

One way to achieve this would be to keep the AST around after initial parsing. Let me give you an idea of how it might work for the following list comprehension:

customers = DataProvider()
data = [(c.Name, c.Value) for c in customers if c.City == "London"]

The DataProvider would provide a __query__ method that takes an AST. For every list comprehension the compiler would have to keep the AST for the expression if c.City == London. Python would then check if customers has __query__, if it does then it will be called with the AST. The AST could have also have the compiled version of the expression (needed for ordinary list comprehension). This would effectively be a lambda taking c as an argument.

Of course this is only interesting if data providers are implemented that could decompose the AST into optimised queries (which is exactly what Microsoft has done with LINQ). The complexity of implementing this, and possible performance costs in list comprehensions, means that it is unlikely to happen in Python.

Still, LINQ is a worthy way of providing a uniform way of querying data and making queries a first class part of the language.

A related question is how can we get this into IronPython. Adding it would mean adding new syntax to Python, which the community (me included) would be very dubious about. Perhaps a compromise similar to the way .NET methods are enabled / disabled on builtin types. Until you do import clr (or import System) the .NET methods are unavailable. A __future__ import would be one way of supporting this - perhaps by agreement with the CPython developers that the corresponding flag will never be used by CPython.

I've recently been wondering about a way of experimenting with adding language features like this. The ANTLR parser has a Python grammar. If you could write an ANTLR AST to Python AST transform, then you could extend the ANTLR Python grammar and still transform to a valid Python AST. Obviously you could only add syntactic sugar to the language, but its surprising how much you can achieve with a bit of sugar!

For example what I'd really like to experiment with is adding syntax for code blocks. The transform phase could transform code block syntax into functions with a compiler assigned name. It would be interesting to see how useful code blocks are in practise.

For buying techie books, science fiction, computer hardware or the latest gadgets: visit The Voidspace Amazon Store. If you're looking for a new techie job, try the Voidspace Tech Job Board. This is part of the Hidden Network of technology and programming jobs.