|
The Common Lisp Object System (CLOS) Metaobject Protocol provides users with access to data objects unlike other programming languages
June 24, 2007
URL:http://www.drdobbs.com/parallel/lisp-classes-in-the-metaobject-protocol/200000266
The first thing to realize about the Common Lisp Object System (and LISP) is that many things that are very much behind the scenes in other languages are availab le as user-accessible data objects in CLOS. For example, in C++, only the compiler has any data structures that correspond to user-defined classes. In CLOS, user-defined classes are represented by objects -- class objects -- that can be examined and manipulated by user programs. This property is sometimes called "introspection."
When you create a class you get a class object back:
(defclass rectangle ()
((height :initform 0.0
:initarg :height)
(width :initform 0.0
:initarg :width)))
#<Standard-Class F00>
The exact contents of the class object depend on the implementation of CLOS.
CLOS was designed in layers. The outermost layer (the one that programmers will typically use) contains the syntax of CLOS, defined by macros and functions such as defclass and defmethod. This layer is called the "user interface layer", but it isn't a user interface in the keyboard, mouse, window, graphics sense; it's a use interface to the operational part of the CLOS language. This user interface corresponds to language definition in other programming languages.
The next lower layer is the connection layer, and it is implemented by functions. The user interface macros expand to calls to function s in this layer; in addition, this layer contains functions that map names to metaobjects. For example, find-class is a function in this layer. The deepest layer is the metaobject layer. Metaobjects are first-class LISP objects that provide the behavior of classes, instances, slot access, generic functions, inheritance, and method combination.
One way to understand this layering is by analogy to cars. You can think of cars as having three layers just like CLOS. The outermost layer is almost like a user interface. This layer is basically all the controls (gas pedal, steering wheel, brakes). The driver interact s with these controls to make his car do things.
The next layer connects the controls to the things controlled. The gas pedal is connected by rods, levers, and springs to the carburetor. The function of this layer is to connect a device designed for easy human use to a device designed to accomplish its function readily. This is like the connection layer.
The deepest layer is the set of devices that make a car perform its function. For example, the carburetor controls the speed of the car by controlling the amount and rate of fuel burned. This is like the metaobject layer.
Changing the way a driver interacts with the car can simply involve the user interface layer; for example, if you want to change the sha pe of the gas pedal or slightly change its placement. For a larger change (such as implementing the gas pedal as a roller on the steering wheel), the interface and connection layer must be changed; that is, a new linkage must be designed and built as well as changing the gas pedal itself.
Finally, the nature of the car itself can be changed by changing the lowest layer. The engine can be replaced with a much more powerful one for amateur racing or trailer pulling. The suspension can be altered to make a road racing car. The back seats and rear sections can be altered to form a pickup truck. The body and undermachinery can be enclosed in a waterproof shell and a propeller added to create an amphibious vehicle. Some of these alterations require changing or a dding things to the connection and user interface layers, but the fundamental nature of the car cannot be changed without changing the deepest layer.
Some changes are just not possible unless the car is thrown away and a new design produced. An Indy car cannot be made from a stock model, no matter how much customization you do. Neither can you create an airplane from a car.
It would be reasonable for an innovative car designer to take the user interface of a car -- the control layout and its functional specification -- and design a different connection and metaobject layer, which would be functionally equivalent to a car but very different underneath. Turbine cars and hovercraft are simple examples.
A lot of CLOS programs will use only the user interface layer, some times dipping into the connection layer. But occasionally the very nature of CLOS needs to be customized, and the metaobject layer must be used. CLOS, along with the Metaobject Protocol, is defined in such a way that implementors are somewhat constrained by having not only the user interface layer but parts of the connection and metaobject layers specified.
The full Metaobject Protocol describes how classes are made and represented, slots are represented, inheritance is implemented, generic functions and methods are made and represented, generic functions invoke methods, how method combinations are represented and used.
To start, we will look at the representation of class objects and get far enough to define customized slots. In the course of our examination of this part of the Metaobject Protocol, we will look at how classes are created and how their slots are accessed. My explanation will be in terms of hypothetical code that creates classes and accesses slots at the accessor level. My code definitions are inaccurate and intended only to give a sense of their actions. For example, for simplicity I will assume that when a class is defined, all of its superclasses are already defined.
Objects that represent classes should provide the following information: the class's name (if any), its direct superclasses, the slots it directly defines, its class precedence list (though this can be computed), the complete set of slots (including inherited ones), its direct subclasses, and the methods defined directly on this class.
Listing 1 shows the information for the rectangle class object (previously defined). With such information, classes can be found by name, inheritance can be determined, instances can be made, and methods can be selected.
Name: rectangle Direct superclasses: none Direct slots: (height weight) Class precedence list: (rectangle standard-object t) Effective slots: (height weight) Direct subclasses: none Direct methods: none
Because class objects are objects, they are instances of classes. Therefore, the class rectangle is itself an instance of a class, called standard-class. Listing 2 shows the definition of standard-class as it would be specified by a defclass form.
(defclass standard-class ()
((name :initarg :name
:accessor class-name)
(direct-superclasses :initarg :direct-superclasses)
(direct-slots :accessor class-direct-slots)
(class-precedence-list :accessor class-precedence-list)
(effective-slots :accessor class-slots)
(direct-subclasses :initform '()
:accessor class-direct-superclasses)
(direct-methods :initform '()
:accessor class-direct-methods)))
If user-defined classes are instances of standard-class, somehow defclass (the user interface macro that defines classes) must eventually run make-instance. To do this, defclass expands into a function that calls a generic function in the metaobject layer.
make-instance invokes the initialization protocol. The initialization protocol lets users specify how to make instances of the classes they defined; therefore it is logical that the initialization protocol would be used to specify how to make classes. This is exactly how classes are made.
By looking at how instances are made, we will understand most of what we should know about class and instance representation. Also, by looking at the expansion of defclass into functions that use metaobje cts, we will see a slice through the three layers and better understa nd their relationship.
The code shows how defclass (user interface layer) expands into a call to an underlying function called ensure-class (connection layer):
(defmacro defclass
(name direct-superclasses
direct-slots rest options)
'(ensure-class ',name
:direct-superclasses
,(canonicalize-direct-superclasses
direct-superclasses)
:direct-slots
,(canonicalize-direct-slots
direct-slots)
:name ',name
,@(canonicalize-defclass-options
options)))
The functions that start with canonicalize- perform the bulk of parsing work by putting the superclasses, slots, and options into a standard format. defclass has two important functions: it creates a class object and it associates that class with a name. canonicalize-direct-superclasses dereferences the class names into class objects, and each slot description is turned into a property list that contains the name of the slot, its initform and initfunction, its accessors separated into readers and writers, and its coalesced initargs.
The function ensure-class calls the generic function ensure-class-using-class (metaobject layer), which takes as arguments the class (or nil), name, and keyword arguments, including the ones we've seen for ensure-class, plus the metaclass and a few other things as keyword arguments. Basically, this generic function calls make-instance on the metaclass and with the proper initialization arguments (Listing 3).
(defmethod ensure-class-using-class
((class-object standard-class)
class-name
rest all-keys
key direct-default-initargs
direct-slots
direct-superclasses
name
(metaclass (find-class 'standard-class))
allow-other-keys)
(let ((initialization-arguments ...))
(cond ((null class)
(let ((class
(apply #'make-instance
metaclass
initialization-arguments)))
(setf (find-class class-name) class)
class)))
...))
We will often see the idiom exemplified by ensure-class calling ensure-class-using-class, and it is important to understand the reason for it. In the idiom, the basic function is specified to take some object and perform an operation on it. But that operation really depen ds on the metaclass of the object and not on the object itself, and so the function calls a generic function that takes an additional argu ment, which is the class of the object. An easier example to understand is slot-value, which takes an object and a slot name and returns the value stored in that slot. But to accomplish this, slot-value must access the representation of instances. As we'll see in tracing how classes are made, the representation of instances is determined by metaclasses, so the real functionality of accessing a slot must cons ult the metaclass. This is done by defining slot-value to invoke slot-value-using-class:
(defun slot-value
(instance name)
(slot-value-using-class
(class-of instance)
instance
name))
Before calling make-instance, ensure-class-using-class computes the initialization arguments from the keyword arguments supplied. make-instance-instance is called with the metaclass as its first argument . That is, an instance is made of the metaclass passed in as the metaclass keyword argument; if no such metaclass is passed in, standard-class is used. Notice that two names are passed in: the required argument classname is used to set the class that find-class returns, and the keyword argument name is passed into make-instance to set the class name. This defines a proper name of the class -- a symbol s is a proper name for a class c if (class-name c) = s and find-class s)=c.
Recall that make-instance is defined as if by the code:
(defmethod make-instance
((class standard-class)
&rest initargs)
(setq initargs (default-initargs class initargs))
...
(let ((instance
(apply #'allocate-instance
class initargs)))
(apply #'initialize-instance
instance initargs) instance))
To finish the story of building a class, we need to look at what al locate-instance and initialize-instance do. Listing 4 shows what allocate-instance could be like.
(defparameter secret-unbound-value (list "slot unbound"))
(defun allocate-slot-storage (size initial-value)
(make-array size :initial-element initial-value))
(defun instance-slot-p (slot)
(eq (slot-definition-allocation slot) ':instance))
(defmethod allocate-instance
((class standard-class) rest initargs)
(declare (ignore initargs))
(allocate-std-instance
class
(allocate-slot-storage
(count-if #'instance-slot-p (class-slots class))
secret-unbound-value)))
allocate-instance allocates the storage used to hold an instance. That is, this function determines how the storage for an instance is carved out of memory. Notice that allocate-instance is a generic function that depends on metaclasses.
allocate-instance calls allocate-std-instance, which builds a defstruct-like structure with two structure slots: one slot contains the class and the other contains a vector to hold the instance slots. In this code, I have assumed that all slots are local slots, which simplifies the definition a little without loss of generality.
You will notice a curious thing. This method for allocate-instance is used to allocate storage for instances of user-defined classes, but because standard-class is an instance of itself, it is also used to allocate storage for classes. In practical terms, this is not that weird; it just means that storgate for class objects is laid out exactly the same way as storage for ordinary instances. However, an inherent circularity occurs here: an instance cannot be made until its class exists, but the class of standard-class is itself. Luckily, only the implementors of CLOS need to worry about this, because it is really a problem only when a CLOS system is being bootstrapped -- kind of like picking yourself up by your belt and flying.
More interesting than allocate-instance is initialize-instance. When we discussed the initialization protocol, we saw what the purpose of the initialization protocol was and how initialize-instancefits into it. Listing 5 shows an after method for initialize-instance on standard-class that initializes the class object allocated by allocate-std-instance. It is an after method to follow the CLOS convention for methods on initialize-instance. Reading the code, we see it performs the following steps. First, it determines the superclass objects, which are either passed in or defaulted to standard-object. Next, it installs the newly defined class as a subclass of the superclasses. Then it takes each slot property list and turns it into a slot definition metaobject. The slot accessor methods are then defined, and finally inheritance-related actions are performed.
(defmethod initialize-instance :after
((class standard-class)
key direct-superclasses direct-slots
allow-other-keys)
(let ((supers
(or direct-superclasses
(list (find-class 'standard-object)))))
(setf (slot-value class 'direct-superclasses) supers)
(dolist (superclass supers)
(push class (slot-value superclass 'direct-subclasses))))
(let ((slots
(mapcar #'(lambda (slot-properties)
(apply #'make-direct-slot-definition
slot-properties))
direct-slots)))
(setf (slot-value class 'direct-slots) slots)
(dolist (direct-slot slots)
(dolist (reader (slot-definition-readers direct-slot))
(add-reader-method
class reader (slot-definition-name direct-slot)))
(dolist (writer (slot-definition-writers direct-slot))
(add-writer-method
class writer (slot-definition-name direct-slot)))))
(finalize-inheritance class)
(values))
The details of slot-definition meta-objects are not important. Let 's look at finalize-inheritance. In a real CLOS implementation, finalize-inheritance could not be called from within the initialization p rotocol because a class's superclasses might not be defined at that point. The definition of finalize-inheritance is:
(defmethod finalize-inheritance
((class standard-class))
(setf (class-precedence-list class)
(compute-class-precedence-list class))
(setf (class-slots class)
(compute-slots class))
(values))
finalize-inheritance first computes the class precedence list and stores it in a slot in the class object. Next, it computes effective slot-definition metaobjects, which describe each slot whether direct or inherited. This computation considers the slot inheritancles of CLOS. Listing 6 shows the method for compute-slots, which makes a lis t of the effective slot definitions. Each effective slot definition is computed by gathering all the direct slot definitions for each slot name into a list in the most to least specific order (according to the class precedence list) and passing that list along with the class to compute-effective-slot-definition. compute-effective-slot-definition simply makes an object that represents a slot definition with en tries representing its name, initform (and initfunction), initargs, and allocation type. mapappend is like mapcar except that the results are appended together.
(defmethod compute-slots ((class standard-class))
(let* ((all-slots (mapappend #'class-direct-slots
(class-precedence-list class)))
(all-names (remove-duplicates
(mapcar #'slot-definition-name all-slots))))
(mapcar #'(lambda (name)
(compute-effective-slot-definition
class
(remove name all-slots
:key #'slot-definition-name
:test-not #'eq)))
all-names)))
(defmethod compute-effective-slot-definition
((class standard-class) direct-slots)
(let ((initer (find-if-not #'null direct-slots
:key #'slot-definition-initfunction)))
(make-effective-slot-definition
:name (slot-definition-name (car direct-slots))
:initform (if initer
(slot-definition-initform initer)
nil)
:initfunction (if initer
(slot-definition-initfunction initer)
nil)
:initargs (remove-duplicates
(mapappend #'slot-definition-initargs
direct-slots))
:allocation (slot-definition-allocation (car direct-slots)))))
The important thing to see is that each slot has an entry in a list ; the relative position of an effective slot-definition entry in that list will determine where in the vector the slot is located. Note that we have not yet seen how slots are accessed. What we have seen is how the class object stores information that is used to access slots, and we've also seen how defclass eventually makes an instance of standard-class, which is a metaobject.
Now let's look at slow-value to see how the machinery in place will accomplish this. It is important to remember that a real CLOS imple mentation will not use a mechanism as inefficient as what I'll show for slot access.
The first ingredient in slot access is a function named slot-location, which takes a class and a slot name and finds the relative position in the list of effective slot definitions for that slot. Listing 7 shows the code. slot-location simply finds the index of the slot whose name is slot-namein the list of effecitve slot definitions stored in the class. Only local slots are checked, because only local slots have storage allocated in each instance. (Check the arguments passed to allocate-slot-storage in Listing 4).
(defun slot-location (class slot-name)
(let ((slot (find slot-name
(class-slots class)
:key #'slot-definition-name)))
(if (null slot)
(error "The slot ~S is missing from the class ~S."
slot-name class)
(let ((pos (position slot
(remove-if-not #'instance-slot-p
(class-slots class)))))
(if (null pos)
(error "The slot ~S is not an instance~@
slot in the class ~S."
slot-name class)
pos))))))
The next ingredient is slot-value (Listing 8), which just calls slot-value-using-class. The method for slot-value-using-class on standard-class finds the location of the slot in the class object, gets the slots vector from the underlying instance representation, does an svref via slot-contents to get the value, checks the value to determine whether the slot is bound, and returns the value if all is well. (self slot-value) is similar.
(defun class-of (x)
(if (std-instance-p x)
(std-instance-class x)
(built-in-class-of x)))
(defun slot-contents (slots location)
(svref slots location))
(defun slot-value (object slot-name)
(slot-value-using-class (class-of object) object slot-name))
(defmethod slot-value-using-class
((class standard-class) instance slot-name)
(let* ((location (slot-location (class-of instance) slot-name))
(slots (std-instance-slots instance))
(val (slot-contents slots location)))
(if (eq secret-unbound-value val)
(error "The slot ~S is unbound in the object ~S."
slot-name instance)
val)))
This odyssey might have seemed a little fast, so let's review what we have seen. The user interface macrodeftclass expands into a function (ensure-class) that makes an instance of standard-class (such in stances are themselves classes) and associates a name with the resulting class. Making an instance of standard-class is part of the metaobject level. Making an instance of standard-class goes through the n ormal initialization protocol, and we saw how the initialization protocol is specialized for making instances of standard-class.
Among the information stored in classes are descriptions of each slot, and these descriptions are used to access slots and create subclasses (by helping create effective slot definitions). We saw that the definition of slot-value uses the information stored in the class to determine where a slot is stored.
Because of properties like this, it is often said that a metaclass determines the representation of instances of its instances, while a class determines the structure (set of slot names) of its instances. Notice that it is the definition of the particulars of the initialization process for instances of standard-class that determines how the ultimate instances are represented. The class defined by the user's defclass stores the set of slots and holds the mapping from slot name to slot representation. Accessors (readers and writers) use slot-value, but I did not show the details, because that involves und erstanding generic functions and methods.
All this flamboyant machinery is irrelevant (except for teaching purposes) unless we find a way to use it to introspect a running CLOS program or to modify or customize CLOS. We will look at an extension to CLOS that can be made by altering the machinery we've seen so far.
But first, I need to point out something about the machinery that I left out. If a defclass form contains an explicit: metaclass class option, the expansion to ensureclass will capture it and pass it on to ensure-class-using-class. If the user does not supply the metaclass, ensure-class-using-class will default the metaclass to standard-class.
The extension we will examine is to provide a metaclass that suppor ts dynamic slots. A dynamic slot is one whose storage is allocated when the slot first becomes bound instead of at instance-creation time . This is useful if an application writer is defining very large classes, but only a small fraction of the slots will be bound in a given instance at any given time.
For example, suppose that, in a simulation, a class is defined with 10,000 slots, but due to the semantics of the simulation every instance has only 10 of the slots bound at a given time. It would be a pity (except to the DRAM manufacturer) to have to have each instance require 40 kbytes when only 40 bytes are needed.
The extension will be used like this by the user:
(defclass c (s1 s2 s3)
(sl1 sl2 ... sln)
(:metaclass all-dynamic-slots-class))
where sl 1..sln is a long list of slots, each of which is a dynamic.
This problem will be solved by adding an implementational option called dynamic slots that stores certain types of slots in a secondary, sparse data structure. This data structure will be associated with each relevant instance, and accessing dynamic slots will be slower but more space efficient if some dynamic slots are unbound.
This will be accomplished in two steps. The first will be to make a subclass of standard-class called dynamic-slot-class that lets the slot option :allocation :dynamic appear and which will create the secondary storage. The second step will be to make a subclass of dynamic-slot-class called all-dynamic-slots-class that implements the full behavior. For the first step, we will make a subclass of standard-class:
(defclass dynamic-slots-class
(standard-class) ())
When we are through, we will be able to write defclasses like this:
(defclass c ()
((slot1 :allocation :dynamic)
slot2 )
(:metaclass dynamic-slots-class))
The sparse storage facility will be based on a hash table that associates each dynamic-slot instance with an alist of its dynamic slots. Let's look at the hash table functions first (Listing 9). allocate-table-entry simply puts an entry in the table for an instance. read -dynamic-slot-value extracts the alist for an instance and returns the entry in that alist for the supplied slot name, if the entry is present, and signals an error otherwise. write-dynamic-slot-value adds or updates the entry for a slot value of an instance.
(let ((table (make-hash-table :test #'eq)))
(defun allocate-table-entry (instance)
(setf (gethash instance table) '()))
(defun read-dynamic-slot-value (instance slot-name)
(let* ((alist (gethash instance table))
(entry (assoc slot-name alist)))
(if (null entry)
(error "The slot ~S is unbound in the object ~S."
slot-name instance)
(cdr entry))))
(defun write-dynamic-slot-name (new-value instance slot-name)
(let* ((alist (gethash instance table))
(entry (assoc slot-name alist)))
(if (null entry)
(push `(,slot-name . ,new-value)
(gethash instance table))
(setf (cdr entry) new-value))
new-value)))
Now that we know what we're aiming at, let's customize CLOS. Suppose the user defines a class C, which is an instance of dynamic-slots-class; then when the user makes an instance of C, an entry must be placed in the hash table. The code that accomplishes this is:
(defun dynamic-slot-p (slot)
(eq (slot-definition-allocation slot) ':
dynamic))
(defmethod allocate-instance
((class dynamic-slot-class))
(let ((instance (call-next-method)))
(when (some #'dynamic-slot-p
(class-slots class))
(allocate-table-entry instance)))
instance)
First, the next most specific method for allocate-instance is called to create the basic storage. If there are some dynamic slots, an entry is placed in the hash table. Finally, the instance is returned. The next most specific method is almost certainly the one on standard-class, but it hardly matters since the generic function allocate-instance is specified to return an instance -- notice that we have obey ed this dictum as well. Refer to Listing 4 to see that this method creates storage only for local slots.
Now we can add a method to slot-value-using-class:
(defmethod slot-value-using-class
((class dynamic-slot-class)
instance slot-name)
(let ((slot (find slot-name
(class-slots class)
:key #'slot-definition-name)))
(if (and slot (dynamic-slot-p slot))
(read-dynamic-slot-value instance
slot-name)
(call-next-method))))
This method finds the effective slot definition and checks whether it is a dynamic slot; if so, it looks up the value in the hash table, and, if not, it calls the next most specific method, which is the one in Listing 8.
We've left out the methods for slot-boundp and such, but they are similarly defined. Note that a dynamic slot is unbound in an instance if the alist for that instance contains no entry for that slot. Therefore, slot-makunbound must remove the entry from the alist.
Finally, we can take the last step to make a subclass of dynamic-slots-class that renders dynamic every slot:
(defclass all-dynamic-slots-class
(dynamic-slots-class) ())
The only thing we have to do is add a method to compute-effective-slot-definition to force the allocation to be dynamic:
(defmethod
compute-effective-slot-definition
((class all-dynamic-slots-class)
direct-slots)
(let ((slot (call-next-method)))
(self (slot-definition-allocation
slot) ':dynamic)
slot))
This method also uses call-next-method to perform the substantial work while only incrementally extending the behavior. This technique is common in object-oriented programming.
In actuality, the Metaobject Protocol is not specified by providing sample code the way I did. The specification lists functions and generic functions, specifies the conditions under which they call each other, lists constraints on user definitions and customizations, and describes invariants that must be maintained.
Courtesy AI Expert
Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.