Ideally there's a backup image that can be loaded to get the device back online, but that's not always a guarantee. Some vendors seem to think it's OK to make such substantial changes to their firmware from release to release that backing out to a previous known-good version is not possible once the upgrade has reached a certain point. In some instances, this is unavoidable; still, hardware vendors should provide intermediate code to bridge the gap and allow for recovery in the event of a failed or aborted upgrade. Some vendors do this; many do not.
Because of this, you must fully inspect the release notes and changelogs of any firmware you intend to upgrade. If you come across any element of an upgrade that isn't fully described or doesn't quite make sense, research it and leverage your support contract to verify you are covered before starting the procedure.
The right way to upgrade
This is not to say that upgrading firmware is a bad idea -- it's not. But blithely upgrading firmware without a clear and useful reason to do so can easily cause more problems than it's worth. Sure, it's nice to see that you're running the latest and greatest revision, but at least in your infrastructure, it's essentially untested code and should be treated as such until it puts in enough miles to become trusted.
Another problem caused by not upgrading firmware: Eventually you get so far behind the current revision that you need to perform several upgrades in a row to get to the version addressing a new issue or providing a newly required feature. Yes, this is a pain and generally requires several reboots, but at least you get it out of the way all at once. On such occasions, it's not a bad idea to have a support tech on the line, especially during touchy upgrades. No matter how much time you invest searching a vendor's knowledgebase or related forums, there's often an internal solution not published anywhere. I ran into one such issue recently when a certain virtualization vendor's own scripts (taken directly from their knowledge base) were causing more problems than they were solving. A call to support revealed the right script -- a script that appears nowhere on the Internet, as far as I can tell.
In the end, when it comes to firmware, the best approach is to split the middle. Be cognizant of the need to update firmware from time to time, but do not think that it's a hard and fast requirement. In the face of functionality problems, security risks, or the need for specific features, do the research, plan your recovery strategy, and perform the upgrade.
Lacking those elements, there's no need to be quite so proactive. It's sometimes best to let sleeping dogs lie.
This story, "Think twice before upgrading core firmware," was originally published at InfoWorld.com. Read more of Paul Venezia's The Deep End blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.