If there's a mantra for deep IT, it might go like this: When something is working right, leave it be, lest it change its mind. That pretty much encompasses everything from the switchport on up, for every layer of the stack, and then some. However, there's a time and a place to start messing around with an otherwise perfectly functional system. In fact, you really should break stuff on purpose when the opportunity presents itself.
The primary driver for this bizarro IT behavior is obviously new builds. If you're standing up a new architecture or framework -- be it server, network, storage, database, whatever -- and it works perfectly the first time, you shouldn't stop there and call it good. Once the production flag flips on that build, you'll never have the opportunity to torch the system again until it's replaced or it breaks horribly.
[ Also on InfoWorld: Let loose your Chaos Monkey. | How to become a certified IT ninja. | Get expert networking advice from InfoWorld's Networking Deep Dive PDF special report. | Subscribe to the Data Center newsletter to stay atop the latest tech developments. ]
Now is the only time when you can run roughshod over your setup and configs, yank pieces out and put them back in, and see what happens when you twiddle knob X and press button B. Sure, you could do that in a dev environment, but depending on your task, you may not have a dev environment -- at the least, a replica of the production system is not truly the production system.
A case in point might be a new storage array. It's big and burly, with lots of bandwidth and disks, as well as redundant everything. You cable it up, flip on the power, log in, assign addresses, create a few volumes, and whatnot. You might have to dig a little deeper to make sure it's configured the way you planned, but you're probably going to find that, ultimately, storage is a simple creature, and everything just works the first time, almost by accident.