The Physical Layer
The physical layer characterizes individual signals and stipulates their behavior on transmission and receipt. It operates on the phit granularity; recall that this is the unit of information that the physical layer transfers per clock edge. This corresponds to 20 bits of information (for a link operating at full width). Note that the additional bits have meaning, in both data packet headers as well as in other packet types.
The physical layer transmits each bit in an individual lane implemented as a high-speed, DC-coupled differential pair. A uni-directional link consists of 20 such lanes plus an additional forwarded clock lane. Since each lane requires two physical wires, a single link requires 42 signals. Components are typically connected with two links, one in each direction, so that each component can simultaneously send to and receive from the other. At each end, the agent refers to its lanes as either Tx for transmission, or Rx for receipt. This coupling is dubbed a link-pair, also referred to as a full link, port (at each end), or connection. Thus a link-pair contains 84 wires, versus approximately 160 for the FSB. Furthermore, Intel QPI requires no side-band signals (additional signals beyond the ones comprising the link).
Electrically, the physical layer handles the conversion of digital data to and from high-speed analog signals, and drives data transmission in synchronization with the clock. As mentioned earlier, it supports a double data rate at two times the forwarded clock frequency, for signaling speeds of up to 6.4 GT/s.
To ensure proper data recovery at these high data rates and to ease some PCB layout burdens, the physical layer also implements waveform equalization, data recovery and deskew circuits, lane reversal, and polarity inversion. Waveform equalization adjusts the electrical characteristics of an analog signal to account for signal degradation along the interconnect channel between the devices. Signal integrity simulation identifies appropriate equalizer circuit tap settings that pre-shape the generated signal so that after it changes along the wire, the receiver observes the intended waveform. Next, primarily due to the routing of individual lanes, some skew occurs between the clock and the data lanes. Intel QPI data recovery circuitry allows several bit times of such skew, and performs deskew per lane. Finally, lane reversal is a PCB layout friendly feature which allows crisscrossing of lanes from Tx to Rx (lane 0 connects to lane 19 and vice versa) and therefore avoids routing congestion on the PCB. During initialization, two components connected with Intel QPI train with a pre-defined handshake sequence to detect and account for any lane reversal. Intel QPI also supports polarity inversion, which allows the PCB layout to swap the positive and negative signals in the differential pair defining a given lane.
The logical aspect of the physical layer interfaces with the link layer. The link layer provides and consumes groups of phits called flits; the physical layer manages this conversion. To reduce power consumption or to work around failures, the physical layer also allows width reduction. Its 20 physical lanes are divided into four quadrants of five lanes each, and (as a product optional feature) a link can operate at full, half, or quarter width. Intel QPI ports automatically detect this during link initialization, which can be re-triggered by a reliability event that may have made a lane unusable. When this occurs, the link optionally performs self-healing. This consists of dynamically negotiating at both ends to determine the new width (half or quarter) and deciding which quadrants will form the reduced width link. If a hard error disables the clock lane, the physical layer may also optionally execute clock failover, where it remaps the clock to a data lane and then dynamically reduces the link to half width (i.e. 10 lanes in each direction). Reduced bandwidth is the only consequence of either case. There is no loss of error detection or recovery capability and all message traffic can proceeds as before, just at a slower rate.
Thus the physical layer also controls how many phits are needed per flit, and how to map phits to the available physical lanes. Likewise, the physical layer is designed to accommodate a certain rate of bit errors due to noise and jitter. To keep this low, the link periodically retrains its usage of all of these features.
To verify link operational integrity, Intel QPI provides Interconnect Built-In Self-Test (IBIST), and several loop-back testing modes which allow self-diagnosing of the link at full speed without requiring external probes.