Channels ▼


Taming Python

Implementing, Deploying, Managing

Let's break it down to digestable pieces and see how to implement, deploy, and manage it:

  1. Intercept somehow every attribute setting on every object of interest.
  2. Determine if the attribute is being set by code in __init__()/__setstate__() or called indirectly from __init__()/__setstate__()
  3. If the policy is violated raise an exception
  4. Find an easy way to attach the policy checking code to every target object
  5. Performance. All these checks can take a serious toll. The user should be able to turn it off.

Intercept Every Attribute Setting. Python provides the answer through the __setattr__() magic method. This method is defined on the base object and performs the assignment operation for all objects. If you override it in your class then every attribute assignment goes to this method. Here is a simple example: Class X defines a __setattr__ method that just prints a message when an attribute is set on an instance of X. This happens when a method of X itself like __init__() is trying to set an attribute called 'a' and also when external code is trying to set a 'b' attribute on an instance of X:

class X(object):
  def __init__(self):
    self.a = 3
  def __setattr__(self, name, value):
    print 'Trying to set', name, 'to', value 

x = X()
x.b = 5

assert not hasattr(x, 'a')
assert not hasattr(x, 'b')


Trying to set a to 3
Trying to set b to 5

If you actually want the attribute to be set, you need to call object.__setattr__(self, nsme, value).

Determine the Origin of set Attribute Attempts. The goal is to allow setting attributes only in code originating from __init__() or __setstate__(). You can do it in several ways. For example, you can examine the call stack and verify that it contains __init__()/__setstate__(). But, this is complicated, expansive, and potentially brittle (e.g., you can set an attribute of object x from the __init__() of object y, which will violate the rules). A better way is to turn on a flag at the beginning of __init__()/__setstate__() and turn it off at the end. The special flag is going to be just an attribute called _can_add_attributes. The __setattr__() method will check if the object has that attribute and will allow adding new attributes only if the attribute exists.

Raise an Exception If the Policy Is Violated. That's pretty easy. Whenever a new attribute is set, the __setattr__ will check if the object already has the attribute. If not, it will check if _can_add_attributes is present; if not, it means the policy has been violated and just raise an exception.

Find an Easy Way to Attach the Policy Checking Code to Every Target Object. This isn't easy. The idea is to do it as unobstrusively as possile. If you require people to add a special code in every method they will rebel, forget or more like both. The trick is to modify their classes dynamically. In Python 2.6 and up class decorators are the ticket. In Python 2.2 though 2.5, meta-class will get you there. I will show both techniques. Class decorators are easy to apply. meta-classes can be applied just by deriving from a base class or even better -- a mixin.

Make Sure Performance Is Not Hindered By All This Uber-sophisticated Mechanism. Python is slow. That's not new. There are various ways to speed it up. The best way is always to make it do less. You may consider running all your tests with the the lock attribute checking turned on and then in production turn if off. In Numenta, I use an environment variable to control the lock attribues mechanism. If the environment variable NTA_DONT_USE_LOCK_ATTRIBUTES is defined then the lock attributes mechanism is not engaged and you don't pay for it (other than the one-time check if the environment variable is defined). This way, by default all developers can be confident that they don't violate the lock attributes principle by accident, but in the production environment the environment variable is defined and there is no overhead.

Demo Time

Here is a little class A that that uses the lock attributes mechanism. It simply subclsses a mysterious class called LockAttributesMixin imported from the lockattributes module (which I'll talk about soon):

from lockattributes import LockAttributesMixin

class A(LockAttributesMixin):
  def __init__(self):
    self.x = 777
  def foo(self):
    self.y = 888

The A class defines an 'x' attribute in its __init__() method, which is fine and a 'y' attribute in its foo() method, which violates the lock attributes principle. Let's see what happens if you try to invoke foo().

a = A()

print 'a.x = ', a.x

except Exception, e:
  print ' failed:', str(e)


a.x =  777 failed: Attempting to set a new attribute: y

The result of invoking was an exception with a very informative mesage that says exactly what went wrong (Attempting to set a new attribute: y).

If you try to set a new attribute on a from the outside you will get the same result, but setting 'x' that was defined in __init__() is okay:

  a.z = 999
except Exception, e:
  print 'a.z = 999 failed:', str(e)
print 'Setting a.x to 444'
a.x = 444
print 'a.x = ', a.x


a.z = 999 failed: Attempting to set a new attribute: z
Setting a.x to 444
a.x =  444

This is exactly how lock attributed should behave -- applied easilly just by subclassing some base class, detect attempts to set new attributes outside of __init__()/__setstate__(), and raise informative exception when a violation occurs.

Let's see how it works. All the magic is implemented in a single file -- This file and a test program that uses it are available for download here. The module can be used as is (no external dependencies) and the that contains the last example with the Circle class (described later in this article).

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.