Channels ▼
RSS

Tools

Taming Python


The _allow_new_attributes() Decorator

Decorators are a great Python feature. They let you replace or augment a function or a method with a different function or method (and much more). Very often decorators are used to run some code before and after calling the original method (think "aspect-oriented programming"). Here is a trivial example that just prints 'before' and 'after'. The decorator function called before_after() takes the original function f as argument and returns a new (nested) function called internally 'decorated' that prints before/after calling the original function f. This decorated function will quietly replace the original function:


def before_after(f):
  def decorated(*args, **kwargs):
    print 'before'
    f(*args, **kwargs)
    print 'after'
  return decorated

You apply a decorator to a function/method by writing @<decorator name> before the definition. Here I apply the before_after decorator to two simple functions a() and b() that print 'a' and 'b', respectively:


@before_after
def a():
  print 'a'

@before_after
def b():
  print 'b'

Calling a() and b() results in the following output:


a()
print
b()

In practice, the @ decorator syntax is equivalent to applying the decorator function directly:


def c():
  print 'c'

c = before_after(c)  
c()

Output:


before
c
after

But, it's nicer to use the decorator syntax and you don't need to repeat the function name. If you want to know more about decorators, I recommend Python 2.4 Decorators.

The lockattributes module defines such a decorator called _allow_new_attributes(). This decorator will be used to decorate the __init__() and __setstate__() methods of target classes to set a special attribute called _CanAddAttributes.


def _allow_new_attributes(f):
  """A decorator that maintains the attribute lock state of an object

It coperates with the LockAttributesMetaclass (see below) that replaces the __setattr__ method with a custom one that checks the _canAddAttributes counter and allows setting new attributes only if _canAddAttributes > 0. New attributes can be set only from methods decorated with this decorator (should be only __init__ and __setstate__ normally). The decorator is reentrant (e.g. if from inside a decorated function another decorated function is invoked). Before invoking the target function it increments the counter (or sets it to 1). After invoking the target function it decrements the counter. When the counter reaches 0, it is removed.

  
  """      
  def decorated(self, *args, **kw):
    """The decorated function that replaces __init__() or __setstate__()
    
    """    
    if not hasattr(self, '_canAddAttributes'):
      self._canAddAttributes = 1
    else:
      self._canAddAttributes += 1
    assert self._canAddAttributes >= 1
    
    # Save add attribute counter
    count = self._canAddAttributes
    
    # Run the original function
    f(self, *args, **kw)

    # Restore _canAddAttributes if deleted from dict (can happen in __setstate__)
    if hasattr(self, '_canAddAttributes'):
      self._canAddAttributes -= 1
    else:
      self._canAddAttributes = count - 1
      
    assert self._canAddAttributes >= 0
    if self._canAddAttributes == 0:
      del self._canAddAttributes
     
  return decorated

That's a lot of non-trivial code so analyze it piece by piece. The general structure of the _allow_new_attributes() decorator is just like the before_after(): It takes the original function f as an argument and returns another function called decorated that does some stuff before calling the original function f and then does some stuff after calling. The only difference is what kind of stuff happens before and after. The main idea is to have the '_canAddAttributes' attribute available on the target object (accessed through 'self') during the time __init__() and __setstate__() are called and not exist after they are finished. The reason _canAddAttribute is an integer that is incremented and decremented instead of a simple True/False boolean is that there could be nested calls to base classes. For example, consider the following two classes:


from lockattributes import LockAttributeMixin

class A(LockAttributesMixin):
  def __init__(self):
    ...
     
class B(A):
  def __init__(self):
    A.__init__(self)
    self.x = 5
    
b = B()

When a new b object is created the following calls are executed:


B.__init__() (because a new B object is created)
  A.__init__() (because B.__init__() calls A.__init__())
  self.x = 5

Both A and B are monitored for locked attributes. Class A direcly subclasses the LockAttributesMixin and class B indirectrly subclasses it by sublassing A. Now, suppose _canAddAttributes was just a boolean attributes. Before B.__init__() is executed will be set to True. B.__init__() will call A.__init__() and before a.__init__() starts _canAddAttributes will be set to True again. A.__init__() will execute and when it is done _canAddAttribute will be set to False! But, now the statement self.x = 5 (comes after the call to A.__init__() inside B.__init__()) will fail. The solution is to maintain a count and increase/decrease it appropriately. As long as the count is greater than 0, it is okay to add new attributes.

Python Meta-classes

This is going to be confusing. A meta-class is the class of a class. In Python every object has a class and you can find it out by querying its __class__ attribute:


>>> x = 5
>>> x.__class__
<type 'int'>

>>> a = A()
>>> a.__class__
<class '__main__.A'>

Now, classes are objects too. Yes, in Python classes are also objects. If they are objects, then they must have a class. Indeed, they have. The class of every class is by default 'type':


>>> int.__class__
<type 'type'>
>>> int.__class__
<type 'type'>
>>> A.__class__
<type 'type'>
>>> type.__class__
<type 'type'>

Yes, 'type' is also a class (and an object) and it is also its own class. Is that cool or what? Before you try to figure out all the relationships between objects, classes and meta-classes in Python and rip the fabric of the universe let's move on to class instantiation. Normally, this is done implicitly when the interpreter sees a class definition in a module for the first time. The 'type' class is able to instantiate any class. Python allows you to attach a different meta-class to a class. That buys you the capability to modify almost anything about the target class. For example, you can add new methods to a class or decorate an existing method. The meta-class called M adds a new method called hello() that prints 'hello' and in addition it decorates every method in the class with the before_after() decorator. All the work is done in the __init__() method of the meta-class M. The method accepts as first argument the class instance (called 'cls' instead of the conventional 'self' to remind you that the instances of the meta-class are classes), then the name of the target class, its bases and finally and most importantly a dictionary of all its attributes, which include its methods. To add or modify functions you can either add them directly (see the 'hello' method) or use setattr(). Don't try to modify the dict directly because it doesn't modify the class itself. The M meta-class checks the type of each attribute in the dict and if it is a function (types.FunctionType) then it decorates with with the before_after decorator:


import types

def hello(self):
  print 'hello'

class M(type):
  def __init__(cls, name, bases, d):
    cls.hello = hello
    for name, value in d.items():
      if isinstance(value, types.FunctionType):
        setattr(cls, name, before_after(value))

Now, that we have a meta-class let's attach it to some class. This is done by setting the __meta-class__ attribute of the target class or by subclassing a class that has an attached meta-class. All subclasses of a class with attached meta-class are modified by the meta-class.


class X(object):
  __meta-class__ = M
  
  def foo(self):
    print 'foo'
    
  def bar(self):
    print 'bar'

Instances of x gain an additional 'hello()' method and all their original methods are decorated with before_after:


x = X()
x.foo()
print '-' * 10
x.bar()
print '-' * 10
x.hello()

Output:


before
foo
after
----------
before
bar
after
----------
hello

That's how meta-classes work their magic and can completely modify the behavior of their target classes. Now, let's look at the actual LockAttributesMetaclass


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video