The Routing Layer

The routing layer directs messages to their destinations. Like the protocol layer, it works with packets. A packet is a single protocol-level message, which the link layer manipulates as an integral number of flits. Every packet includes a destination identifier. In its simplest implementation, the routing layer just looks up this identifier in its routing table, a map of which link to take to reach each possible destination. As the routing table depends on the topology, firmware populates it during system initialization. More complex systems could take advantage of the routing layer to create multiple paths to a given destination. In this case the routing layer would take more responsibility for load balancing, perhaps making dynamic decisions based on traffic observations. It further provides the potential for dynamic partitioning and reconfiguration into multiple smaller systems sharing the same physical resources. This ability can help robustness: when software or firmware detects a potentially failing link in the highly reliable Intel Xeon processor line or Intel Itanium processor family, it uses the Intel QPI quiesce feature to drain the system and reprogram the routing tables before resuming normal traffic.

Although collectively the routing layer at all intermediate nodes eventually directs a packet from its initial source to its final destination, at a given node, the routing layer acts on a point-to-point basis. That is to say, the routing layer at a given intermediate node need only know how to direct an incoming packet to the next node, based on the packet's destination Node ID. It lacks a global picture of the whole topology and all legal routes. It is the transport layer which provides this perspective, or the protocol layer in the absence of a transport layer.

To illustrate this independence, consider a simple implementation of the routing layer. Suppose each Rx contains a routing table, indexed by its destination Node ID. When a packet arrives, the routing layer decodes its destination field, and looks it up in its routing table. Assuming the current node is not the packet's final destination, the routing table identifies the next link that the packet should traverse. The Rx then passes the packet to the Tx of that link. This table could be extended to also output the virtual network on which the packet should be sent. The contents of the table could be populated by BIOS, or other firmware or software familiar with the system topology, at initialization.