Microsoft released patches for Server 2016, 2012R2, and 2012 on Nov. 8 that freeze specific Lenovo servers on reboot. The servers don’t finish the POST process and hang at the Lenovo splash screen. After many complaints, Lenovo issued six new UEFI firmware patches on Nov. 22. The next day, Microsoft altered six of its security patches, including the latest Win10 version 1607 cumulative update, KB 3200970, to add logic bypassing automatic installation of those patches on the affected servers.
The flurry of activity led to numerous missing patches on the Microsoft site for several hours on the afternoon of Nov. 23. We had several reports on AskWoody.com of patches that had simply disappeared -- they weren’t available from the Microsoft Update Catalog. By the end of the day on Nov. 23, all of the patches had returned.
If you’ve already installed the patches, there’s nothing you need to do. If you tried to install the patches and they didn’t appear in Windows Update, WSUS, the Download Center, or the Update Catalog, check again. They’re back.
If you have one of the affected Lenovo System X M5 or X6 servers, you need to get the UEFI firmware update installed and then manually install whichever Windows patch you may have missed. If your server won’t boot, Lenovo has instructions for manually flashing the UEFI firmware. As Lenovo’s fix page solemnly warns, “Replacing the system board will not fix the issue.”
As best I can tell, all of the following Windows patches were pulled, disappeared for a few hours, then re-issued in a modified state so they won’t install automatically on affected Lenovo servers. The KB articles have been modified to say:
Known issues in this update
Some Lenovo servers do not start after this update is installed. Lenovo is aware of this problem and has released a UEFI update to address it. In the interim, Microsoft has changed the detection logic in the update to prevent additional customers from being affected. For more information, see https://support.lenovo.com/us/en/solutions/ht502912.
It’s important to note that the bits in these patches didn’t change on Nov. 23. Only the detection mechanism -- the “metadata” -- was changed:
KB 3200970 - Cumulative update for Windows 10 Version 1607 and Windows Server 2016: Nov. 8, 2016 (this is for Win10 Anniversary Update, as the other versions of Win10 weren’t changed).
KB 3197874 - November 2016 Security Monthly Quality Rollup for Windows 8.1 and Windows Server 2012 R2 (“Group A”).
KB 3197876 - November 2016 Security Only Quality Update for Windows Server 2012 (“Group B”).
KB 3197877 - November 2016 Security Monthly Quality Rollup for Windows Server 2012 (“Group A”).
V1.1 (November 23, 2016) Revised bulletin to announce a detection change for certain servers running Windows Servers 2012, Windows Server 2012 R2, and Windows Server 2016. Affected servers will not automatically receive the security update. For more information about the servers affected by this detection change, see Knowledge Base Article 3193479.
The good news: If you have a Lenovo server that won’t boot, a solution exists. The very good news: If you already have the patches installed, there’s nothing you need to do. Nothing to see here, folks, move along.