Accelerating Rich Data Transport by Using Optical Interconnects
As mentioned previously, data-center network bottlenecks present a significant impediment to efficiently processing data rich sources, regardless of whether they are stored or streaming. While the optimized software infrastructures we described are designed to extract as much performance as possible from traditional data-center networks, innovations in the underlying hardware can also be used to augment existing electrical designs and improve performance. As data centers become increasingly large, with tens of thousands of servers, the communications infrastructure and interconnect become more critical for optimum performance. We seek to leverage the well known high bandwidth capabilities of optical components and transmission over optical fibers to build optical systems that improve data-center performance. In particular, we are investigating optically switched systems and advanced modulation formats [4,5,6,7,8].
While optical network transmission components are already in widespread use, switching still occurs primarily in the electrical domain. As the bandwidth of the links between switching fabrics increases, and more optical interconnects are used, it becomes logical to investigate the possibility of using optical switches, particularly to avoid the costly and power-consuming electrical-to-optical and then optical-to-electrical conversions necessary at the switch interface.
Optical switches have the property of "bit rate transparency," meaning that the data rate of the information being transmitted does not affect the switch performance. This property is very advantageous for scalability; the advantages of optical switching in terms of power/bit and cost/bit increase with increasing data rate.
A limitation of optics, however, is that a commercially available equivalent to random access memory does not exist, although there is considerable research into the field of optical buffering technologies. Further, no optical switches are currently available that reconfigure on packet timescales and are practical (having sufficiently high integration and port counts) for data-center applications. However, optical MEMS switches are currently commercially available with relatively high port counts (currently 100s with development underway for scaling) — although they require relatively long reconfiguration times. To compensate for these long reconfiguration times, we have been investigating a hybrid network architecture that augments a traditional packet-switched electrical network with a circuit-switched optical network [4, 5]. In this architecture, depicted in Figure 4, a small number of rack-to-rack optical links provide high bandwidth for long-lived network flows, while the traditional electrically-switched network handles low-bandwidth, latency-intolerant communication.
This architecture can be effective because the characteristics of many data‑center workloads do not require packet time-scale switching speeds, and the hybrid network may relieve bandwidth bottlenecks and improve performance without increasing cost and power consumption inordinately. In the longer term, highly integrated semiconductor-based optical switches with packet scale reconfiguration times should relieve time constraints and offer more flexibility .
In parallel, we are developing a technology to increase the bandwidth of the optical interconnect. Interconnects based on vertical cavity surface emitting lasers (VCSELs) and parallel optical fibers are the most commercially advanced technology and have shown considerable progress in data rate, power consumption, and packaging technologies. Multi-wavelength solutions offer high bandwidth on a single fiber. This research explores the increasingly interesting option of whether digital signal processing and higher-order modulation formats can increase the transmission data rates from a single laser [6, 7]. This technique can be applied to the different kinds of lasers, and it may be particularly useful when using lower-cost, multimode fiber systems that suffer from intermodal dispersion; thereby, limiting the bandwidth-distance product.
As our world becomes increasingly data rich, new technologies are required to support the applications that process data sources — whether they are stored or streaming. In both cases, cloud-computing technologies provide an infrastructure that enables a large number of users to process shared data sets. However, the bandwidth and/or latency requirements of these applications dictate that special care must be taken when designing systems for these applications. Research undertaken within Intel Labs is beginning to discover the technologies needed to provide high-performance computing on data-rich sources.
 Roy Campbell, Indranil Gupta, Mike Heath, Steve Ko, Michael Kozuch, Marcel Kunze, Thomas Kwan, Kevin Lai, Hing Yan Lee, Martha Lyons, Dejan Milojicic, David O'Hallaron, and Yeng Chai Soh. "Open Cirrus Cloud Computing Testbed: Federated Data Centers for Open Source Systems and Services Research." HotCloud, June, 2009.
 Michael A. Kozuch, Michael P. Ryan, Richard Gass, Steven W. Schlosser, David O'Hallaron, James Cipar, Elie Krevat, Julio Lopez, Michael Stroucken, and Gregory R. Ganger. "Tashi: Location-aware Cluster Management." First Workshop on Automated Control for Datacenters and Clouds, (ACDC), 2009.
 Hrishikesh Amur, James Cipar, Varun Gupta, Gregory R. Ganger, Michael A. Kozuch, and Karsten Schwan. "Robust and flexible powerproportional storage." Accepted to Symposium on Cloud Computing (SOCC 2010), June 2010.
 Madeleine Glick, David G. Andersen, Michael Kaminsky, and Lily Mummert. "Dynamically reconfigurable optical links for high-bandwidth data center networks." Conference on Optical Fiber Communication 2009 (OFC 2009), 22-26 March, 2009.
 Guohui Wang, David G. Andersen, Michael Kaminsky, Michael Kozuch, T. S. Eugene Ng, Konstantina Papagiannaki, Madeleine Glick, and Lily Mummert. "Your Data Center Is a Router: The Case for Reconfigurable Optical Circuit Switched Paths." Hot Topics in Networks (HotNets-VIII) October 22-23, 2009, New York City, NY.
 Yannis Benlachtar, Philip M. Watts, Rachid Bouziane, Peter Milder, Robert Koutsoyannis, James C. Hoe, Markus Püschel, Madeleine Glick, and Robert I. Killey. "Real-Time Digital Signal Processing for the Generation of Optical Orthogonal Frequency Division Multiplexed Signals." Accepted to the IEEE Journal of Selected Topics in Quantum Electronics.
 Yannis Benlachtar, Philip M. Watts, Rachid Bouziane, Peter Milder, Deepak Rangaraj, Anthony Cartolano, Robert Koutsoyannis, James C. Hoe, Markus Püschel, Madeleine Glick, and Robert I. Killey. "Generation of Optical OFDM Signals using 21.4 GS/s Real Time Digital Signal Processing." Optics Express, Volume 17, Issue 20.
 Howard Wang, Ajay S. Garg, Keren Bergman, and Madeleine Glick. "Design and Demonstration of an All-Optical Hybrid Packet and Circuit Switched Network Platform for Next Generation Data Centers." Optical Fiber Communication Conference, 2010.
 Padmanabhan Pillai, Lily Mummert, Steven Schlosser, Casey Helfrich, and Rahul Sukthankar. "SLIPstream: Scalable, Low-latency Interactive Perception on Streaming Data." In the Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV), June, 2009.
 Ming-yu Chen, Lily Mummert, Padmanabhan Pillai, Alex Hauptmann, and Rahul Sukthankar. "Exploiting Multi-level Parallelism for Low-Latency Activity Recognition in Streaming Video." In Proceedings of the ACM Multimedia Systems Conference, February, 2010.
Michael Kozuch is a Principal Engineer and Manager of the Systems Research and Engineering team for Intel Labs Pittsburgh. Jason Campbell is a Senior Researcher at Intel Labs Pittsburgh. Padmanabhan (Babu) Pillai joined Intel Labs Pittsburgh in September, 2003, after finishing his doctoral degree in Computer Science and Engineering at the University of Michigan, where he worked with Professor Kang Shin at the Real-Time Computing Lab (RTCL). Madeleine Glick is a Principal Engineer at Intel Labs Pittsburgh and an Adjunct Professor in the Department of Electrical and Computer Engineering, Carnegie Mellon University.
This article and more on similar subjects may be found in the Intel Technology Journal, Volume 14, Issue 1, "Essential Computing: Simplifying And Enriching Our Work And Daily Life". More information can be found at http://intel.com/technology/itj.