Software Defined Networking (SDN) is one of the big buzzwords of year. VMware has announced their VXLAN (Virtual eXtensible LAN) technology and Microsoft is pushing NVGRE (Network Virtualization using Generic Routing Encapsulation) with their Hyper-V package. What are these technologies and how do they differ?
Both of these solutions are aimed at solving a few problems:
- Virtualization requires Top of Rack switches to handles much larger MAC address tables
- The limitation of 4094 VLANs is not enough for a multi-tenant environment
- The use of STP is inefficient and reduces the number of usable links
- Virtualized workloads need to be portable to efficiently use all resources: compute, storage, and network
|160||Outer Ethernet||Outer Ethernet|
|160||Outer IP||Outer IP|
|64||UDP Header||GRE Header|
|160||Inner Ethernet||Inner Ethernet|
|Variable||Inner Payload||Inner Payload|
|32||Frame Check Sequence||None|
VMware is using UDP as their transport protocol and then adding an additional VXLAN header which contains the VNI information. They also are adding a frame check sequence to the back for checksum purposes. Microsoft, on the other hand, is making use of the GRE protocol rather than traditional TCP or UDP and does not have error checking for the outer packet.
Since VMware decided to use UDP for their transport protocol, that means that any layer 4 aware device will have no trouble interpreting the packet. GRE on the other hand lives in between layer 3 and 4. The packet may encounter a network device that does not know how to interpret the GRE header leading to inconsistent performance or possibly dropped packets. That being said, most modern switching devices and routers support GRE so it’s not much of a consideration. The larger issue to me is the expected packet size. VXLAN packets are going to be 96 bits larger than NVGRE packets, and they are subject to UDP checksum processing. Let’s imagine that you don’t have jumbo packets enabled and the inner payload is relatively sizable. Packets could be truncated or corrupted, which may be part of the need for a checksum. The bigger size and additional processing makes me wonder if VXLAN will be inherently less efficient.
Beyond the packet structure the two standards are almost identical in terms of practical use. Both use multicast on the physical network to distribute broadcast and multicast messages on the virtual network. This will require some IP multicasting magic on the physical side to associate particular multicast addresses to virtual/client network identifiers. The RFC from VMware seems to outline a solution for this tracking, while the RFC from Microsoft mostly leaves that up to the control plane to manage. In Microsoft Server 2012 R2 they have added the Windows Server Gateway piece, which appears to handle some of the control plane functions missing in the RFC. I would not be at all surprised if other vendors develop similar solutions based on the NVGRE spec. The RFC is co-written by people from Intel, Dell, HP, Arista, Broadcom, and Emulex. That’s a lot of big names who would be happy to develop their own solutions. The VXLAN RFC is coauthored by people from Cisco, Arista, Broadcom, Citrix, and RedHat. Again, no slouches in that department. I was a little surprised that Cisco wasn’t on the RFC for NVGRE since they authored the GRE standard.
In the end both of these standards are viable for SDN application, and it remains to be seen how they perform in real life conditions. Based on the RFC and documentation from VMware and Microsoft, there is no clear winner here, making it a less of a factor when picking a network virtualization vendor.
NVGRE RFC: ://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-02
VXLAN RFC: ://tools.ietf.org/html/draft-mahalingam-dutt-dcops-vxlan-04
Windows Server Gateway: ://technet.microsoft.com/library/dn313101.aspx
Director, Cloud Solutions and Microsoft MVP: Cloud (Azure/Azure Stack) & DC Mgmt