One day I received a call from Taj, asking if I could restore a directory from tape. He'd gone to restore a database and it looked like his local backups were screwed up. I said, "No problem," and, as I'd been doing for weeks, restored the file.
Minutes later, I received another call from Taj, asking if I had done the restore. I said that I had. There was a moment of silence, and he said, "I'll be right down."
He walked into my office a few minutes later, clearly irate. "When did Opie ask you to start backing up this server? It was Opie, right?" I told him that the request had been made a few weeks before. He looked at me for a moment, then spun on his heel and headed back to the developer pen. Sensing something might be about to go down (and that my name could be involved), I followed him.
"I know you've been screwing with my database for weeks," Taj literally pointed his finger at Opie, "and now I've lost all of my work." Opie leaned back in his chair, arms folded, and didn't say a word, just rocked in his chair. Taj looked at me, as if for assistance, and I shrugged my shoulders and said, "I just do the backups."
The director of IT was called in to settle the dispute. Over the course of a long, intense conversation, the details came to light.
Apparently, Opie felt like his input wasn't being acted upon and had been working out of his silo. Opie was trying to shoulder the job that Taj had been hired to do, along with the tasks he'd been assigned, because he felt he could do a better job.
Taj would come to work early and would leave before Opie. After Taj left for the day, Opie would back up Taj's files and then restore his own database from the tape backup. When Opie was finished working, he would restore the database from the local backup he'd created of Taj's work and then save his own work as the backup file. The backup job would take his data to tape, so in the event that Taj overwrote the backups, Opie would have a copy of his work.
If you're following who has a backup of their work and who doesn't, what happens next should be a foregone conclusion. The problem came when Taj hit an issue and, rather than work around it, decided it would be quicker to just restore from backup. When he completed the restore, he was looking at a database that he hadn't designed -- it was Opie's project that sat on the screen, staring back at him.
I'm guessing it was due to the fact that he had written a core program used by the company (and he was the only one who really understood it) that Opie wasn't fired. He was removed from the project, and in the rest of my time with the company, I don't think he was ever put in charge of anything else.
What I learned from this experience is that your data doesn't exist unless you have it in three places -- and at least one of those places should be under your direct control.