ESXi 6.0 Update 2 may fail with PSOD!

VMware released ESXi 6.0 Update 2 on 16-MAR-2016. It included nice improvements (VSAN 6.2, HTML5 host client, ...) and bug fixes and a number of customers are already at the latest version.

Those who are quick to update should be aware of a new critical issue logged in the VMware KB 2144968, published originally on 16-APR-2016. Your ESXi 6.0 U2 hosts may fail with Purple Screen of Death (PSOD) if they have VMs with hardware version 11 and if Large Receive Offload (LRO) for VMXNET3 NICs enabled. The purple diagnostic screen would contain output similar to:

0xhhhhhhhhhhhh:[0xhhhhhhhhhhhh]Vmxnet3VMKDevRxWithLock@vmkernel#nover+0x1f2f
0xhhhhhhhhhhhh:[0xhhhhhhhhhhhh]Vmxnet3VMKDevRx@vmkernel#nover+0x2df

‍

LRO reduces the ESXi CPU overhead for TCP/IP network operations. It aggregates multiple incoming packets into fewer but larger packets and then passes them up the networking stack. This way the physical NICs offload some of the work that would be done by the host CPU. LRO is enabled by default on the ESXi hosts.

To sum up, if you have ESXi hosts running 6.0 U2 with LRO enabled and hosting VMs with hardware version 11, you should take quick actions to avoid PSOD.

Use Runecast Analyzer to check if any of your hosts are affected, based on the combination of these 3 conditions mentioned above. If you are already running Runecast Analyzer, ensure that you are at build 1.0.0.27 or higher.

You can then disable hardware LRO for the affected hosts using esxcli:
esxcli system settings advanced set -o /Net/Vmxnet3HwLRO -i 0

VMware released patch ESXi600-201605001 on May 12, 2016 that fixes the issue. If you are not ready to apply the patch, as a work around you can disable LRO.