Peter Christy

Juniper Stratus Becomes Concrete

With a great deal of hoopla Juniper introduced QFabric, a quite revolutionary data center networking offering that they have been alluding to for some time as Project Stratus. Although Juniper doesn't choose to think of it this way, there are clear parallels between QFabric and OpenFlow. In both cases all the decision making is done at the edge of the network (the entry and exit ports) and the control of the network is unified. In the case of QFabric, Juniper has introduced an element called a Node in the form of a top–of–rack switch. In QF 2011 a data center network consists of <100 of these switches that act in concert in a unified way (in the OpenFlow analogy), the OpenFlow controller has been implemented with elements in each of the edge T–o–R switches. There is a second element called the Director that is an X86 based appliance that serves both to help coordinate the activity of the nodes (although dynamically they don't require the Director on most actions) as well as presenting a unified management and control abstraction of the network (not unlike that management appliance in a Cisco UCS or VBlock system). The final element is the fabric proper. In this kind of architecture the interior of the network is basically a data backplane interconnecting the edge switches, and Juniper has built exactly that – a carefully designed and modularized physical network. The Fabric has three important properties: (1) cost and power requirements scale linearly with the number of edge ports in the network (conventional data center network interiors scale exponentially and fail at large scale); (2) the mesh provides feedback so that the entry nodes quickly learn of congestion at an exit node, and can use that feedback to assure that the Fabric proper will never have to drop traffic; (3) the Fabric is fully non–blocking meaning that the only time congestion will occur is if the target exit port is congested. QF is much less vendor–neutral that legacy network elements that can be intermixed. The Nodes must be connected to the Fabric although QF can grow within a heterogeneous network. So using QF is a pretty big decision. However what it offers is exciting enough that it's hard to imagine that it won't make some waves in enterprise and SP data center networking.

"Twist and Shout"

Network Revolutions for Storage Too? The attentive reader will already know that I think that the OpenFlow discussion is one of the most interesting network architecture topics to have come along in a long time (maybe the time is right to do data center network differently and better). Recently, I had a chance to have lunch with our old friend Kevin Brown (from the Inktomi days originally with a lot of entrepreneurial success subsequently) about Coraid, a storage company he joined in the last year. Coraid is an interesting story by itself having been started by Brantley Coile, the technologist in back of the Pix Firewall and Local Director (and then much more) with funding brought in by Chairman Audrey MacLean. Coraid started with the interesting but limited goal of being able to use raw Ethernet to build a storage fabric (eliminating the need for an alternative SAN fabric, without depending on Ethernet advances like the FCoE technology under the Cisco Nexus switch) and they've done that, focusing initially on Linux and research environments. But in today's virtualized data center, Coraid could be a solution with the same kind of benefit that OpenFlow hopes for — a way of simplifying and automating the storage network, while addressing new requirements like real multi-tenancy, without the incredible complexity required using today's SAN technology and its topological limitations. Kevin is obviously breathing a much bigger vision into Coraid, bolstered by A-list Board and advisor additions. Based on his track record to date, it would be foolish to underestimate what might result.

Drew and Woz Meet Andy

As part of our data center networking investigations, we visited with Arastra, Andy Bechtolsheim's latest venture. Andy is one of the most remarkable Silicon Valley fixtures. Not only has he made a lot of money, but along the way he's done some extremely elegant product designs, often doing things simply better than others or doing things that others didn't think could be done (or hadn't thought of). Arastra looks like another winner. It's a 10G switch, and as such is in a category most will never think about but probably use every day. But for those who deal with it directly, it's another really elegant example of Andy's work as the picture below shows:

Arastra's first product is a 48 port 1RU device which is a packaging feat not unlike the Faberge Eggs that the Romanov's were so fond of. Just putting those 48 connectors in a 1U box and still having cooling work is pretty amazing to those practiced in the art. The Arastra switch is also interesting because it's a real deviation from a lot of Andy's early work where he used high complexity custom silicon to create a barrier to others to follow. The Arastra switch has none. Instead it leverages standard parts and creates differentiation through packaging. Steve Wozniak practiced similar art with the Apple II making amazingly clever use of standard parts while others piled on expense to do the same thing. Since hardware isn't the competitive barrier here it must be software. The Arastra switch is designed for the addition of third party software (or the adaptation of existing code into switch functionality). The base operating system of the switch is LINUX and then real-time capabilities are provided by an innovative NetWare like inner environment with a shared state repository providing a lot of the coordination. NetWare was a marvelous software system when Drew Major first created it 20+ years ago in order to make a MHz class PC serve as a high-performance I/O controller (today's multicore CPU's are maybe 5,000 times more powerful). High performance software designers rediscover NetWare regularly. It's sad we don't just teach it in school.

Critical Infrastructure Failures

The freezer compressor for the Safeway across the street failed resulting in the loss of $100,000 worth of food, no small hit to a modest grocery store that runs on razor thin margins. I wonder if they have reconsidered having a more costly redundant system? Earlier this year there was a spate of undersea cable failures in the Middle East, enough and some in strange enough circumstances to lead to speculation that it wasn't simply chance. In the not too distant past undersea earthquakes and landslides lead to multiple cable failures in the Far East. All of these cable failures significantly degraded Internet performance, especially for the time it took to sort out the new routes and stabilize everything. What's the point of all this? Most business executives and government leaders have no understanding of the structure or potential failure modes of the Internet that we all increasingly depend on, just like the manager of the Safeway, I'm betting had any no idea of how the freezer infrastructure was designed until the proverbial shit hit the fan. Even for network experts it's easy to design in the use of "redundant" networks for reliability only to learn under duress that all the providers share a common fiber cable. Should we be more worried about any of this? Is a more serious network failure due to not paying enough attention to risk as likely as a financial meltdown due to the inadequate understanding of the cascade started by sub-prime mortgage securities? Probably.

Cisco / IronPort -- A Spam Inflection Point?

I was going to write a depressed tirade for the newsletter about how little apparent industry cooperation exists against spam. Then, in the middle of my funk, Cisco announced the intention to acquire IronPort. What’s exciting about that (potentially) is Cisco’s dominance in enterprise network (they aren’t allowed to say that but I am) and IronPort’s reputation information. IronPort has exploited reputation information from the beginning and it’s my belief that it provides the most important single factor in practical spam control. What, I ask, would happen if that reputation information was all of a sudden broadly available to Cisco switch and router users? In theory that would be exciting and meaningful, although there are no shortage of reasons why it might not happen, including but not limited to: (1) the acquisition not going through, (2) Cisco really operating IronPort as an autonomous division and not aggressively commingling the technology and (3) the Cisco network guys thinking it’s a bad idea. In the past some of Cisco’s most distinguished engineers have argued eloquently (if wrong) that associating reputation with IP numbers is bad thinking. I guess I’ll just have to wait and see, and keep that depressive tirade fresh and ready to use.