Today I stumbled across a nice RFC draft which proposes a new kind of network topology in data centers (thanks to Omar Sultan for the link on his blog). It’s four days old (at the time of writing) and is proposed by some mayor players in the data center market: it mentions Cisco, Red Hat, Citrix and VMware among others.
It proposes the use of VXLANs, or Virtual eXtensible Local Area Networks, which is basically a tunneling method to transport frames over an existing Layer 3 network. Personally, after reading through it, the first thing that came to mind was that this was another way to solve the large layer 2 domain problem that exists in data centers, in direct competition with TRILL, Cisco’s FabricPath, Juniper’s QFabric, and some other (mostly immature) protocols.
But then I realised it is so much more than that. It comes with 24 identifier bits instead of the 12 bits used with VLANs: an upgrade from 4,096 VLANs to 16.7 million VXLANs. Aside from this it also solves another problem: switch CAM tables would no longer need to keep track of all virtual MAC addresses used by VMs, but only the endpoints, which at first sight seem to be the physical servers only (I don’t think this is a big problem already. The draft claims ‘hundreds of VMs on a physical server’, which I find hard to believe, but with the increase of RAM and cores on servers this may become reality soon in every average data center). It also seems to have efficient mechanisms proposed for Layer 2 to Layer 3 address mapping and multicast traffic. Since it creates a Layer 2 tunnel, it would allow for different Layer 3 protocols as well.
Yet I still see some unsolved problems. What about QoS? Different VMs may need different QoS classifications. I also noticed the use of UDP, which I understand because this does not have the overhead of TCP, but I don’t feel comfortable sending important data on a best-effort basis. There is also no explanation what impact it will have on link MTU, though this is only a minor issue.
In any way, it’s an interesting draft, and time will tell…
Reggle 
Interesting topic!
I’ve been researching the need to eliminate spanning-tree in the datacenter for a while now. On Cisco Live! in London I followed a few seminars concerning this issue and talked to some of the biggest out there like Facebook or Google. The latter solve their issue with Fabricpath, a Trill implementation. When I read the RFC, I believe it’s just another way of dealing with the issue.
In fact Datacenters like Facebook or Google have a large quantity of the same server environments and a relatively simple network setup (several PODS which are Layer3 boundaries). The RFC describes how to connect existing Layer2 PODS over an actual Layer3 network thus extending the Layer2 network over an existing Layer3 design.
I’m also wondering what the future will bring concerning this topic!