Modern physical NICs (pNIC) have several offloading capabilities. If you are running VMware NSX, which is using VXLAN, you could benefit from the VXLAN offloading feature. Using VXLAN offloading allows you to use TCP offloading mechanisms like TCP Segment Offload (TSO) and Checksum Segment Offload (CSO) because the pNIC is able to ‘look into’ encapsulated VXLAN packets. That results in lower CPU utilization and a possible performance gain. But how to determine what is actually supported by your pNIC and the used driver in ESXi?
It is recommended to follow these three steps to fully verify if the VXLAN offload feature you are looking for is supported and enabled.
Step 1: Check the support of the pNIC chipset
Step 2: Check the support of the driver module
Step 3: Check if the driver module needs configuration
The first step is to check the vendor information about the supported features on their pNIC product. Let’s take the combination of a 10GbE Broadcom QLogic 57810 NIC and the VXLAN offload feature as an example. Looking at the datasheet of the QLogic 57810 NIC, it clearly states that VXLAN offloading is supported.
Now that we know the hardware bit supports VXLAN offloading, it is time to check if the driver module does. An easy way to verify this, is by retrieving the driver module details and check the support against the VMware Hardware Compatibility List (HCL). So, we should check what driver module is used for the pNICs. We can do so by listing the NICs within the ESXi host: esxcli network nic list
That will give you the information, from which you can derive the driver. In this case, the bnx2x (Broadcom 10GbE chipset) driver module is used. To get the specific driver information, like the driver and firmware version, you can use the following command: esxcli network nic get -n vmnicX
The output contains the driver information, specifically the driver version (2.713.10.v55.4), that allows you to look up the specific information on the VMware HCL.
The information provided in this screenshot can be used to verify if the pNIC feature is supported in the used driver module. Using the HCL, we should be able to find the driver and what features it supports. It may come in handy to have more information available about the pNIC and the driver used if the HCL has a lot of listings. In that case, you also need the hardware ID properties to make sure you are looking at the correct driver in the HCL:
- Vendor-ID (VID)
- Device-ID (DID)
- Sub-Vendor-ID (SVID)
- Sub-Device-ID (SDID)
To extract that information from your ESXi host, you can use the vmkchdev -l | grep vmnic command. This will list additional hardware ID information about the vmnics.
The given output can be interpreted as follows:
0000:01:00.0 8086:1528 15d9:1528 vmkernel vmnic0
The bold section shows the hardware device properties in the format VID:DID SVID:SSID. When you are in doubt which VMware HCL entry you should choose for your pNIC, this information allows you to identify the unique HCL entry. The bnx2x driver entry in the HCL with the matching driver version shows that VXLAN offloading is supported.
So, why do we need the third step to verify the supported features? This specific example will show you that. The interesting thing about the bnx2x driver module is that certain settings are supported but not enabled by default, in particular the VXLAN offload feature.
esxcli system module parameters list -m bnx2x
The parameters list command shows the driver module for the bnx2x driver module.
In the exemplary screenshot, the parameter enable_vlxan_ofld is stated to be disabled by default. This is the perfect example why you should always check if a manual configuration is required for a feature to be enabled.
Do not automatically assume the driver feature is enabled when it is supported!
More information…
…can be found in the vSphere 6.5 Host Resources Deep Dive book that is available on Amazon!
