I had an interesting experience last week during an office relocation that hammered home some of the fundamental requirements of such a task -- namely, always know where your circuits are.
As far as relocations go, this wasn't that big of a deal. A small remote office, around 150 copper ports, single data T1 tied to the corporate network via MPLS, a single PRI circuit, and a few dozen VoIP phones. No big deal, really.
Four Cisco switches, an aging 2600 router, a few servers, some phone gear, some UPSes and wiring. In and out in a day, right? Well, it should have been.
The building housing the new office suite is a standard small office building. Three floors, a few dozen suites, and five mechanical/wiring closets. The two circuits were installed and verified a month before the actual relocation took place, and all appeared to be well. The wiring company had come in and run all the local office lines, as well as demarc extensions for the PRI and T1 circuits. The wiring was exceptionally neat and tidy, and the extensions were well placed in the small server room. Everything was ready to go.
The morning of the move, we cabled everything up, moved the routers and such from the old location, racked everything, and started to test the circuits. Nothing. Not a sausage. From every viewpoint, it appeared that the extensions weren't run to the NIU in the wiring closet. Okay, no big deal, maybe they weren't patched. Armed with the circuit IDs, I headed out to find the other end of the demarc extension.
That's when I discovered that the building policy dictates that the wiring closets are not accessible after 11am weekdays, and completely inaccessible on the weekends. This was a Thursday at around 1pm. According to the building management, there was nothing they could do about it -- those rooms would remain locked. Our circuits were being held hostage.
So instead of a quick-hit, in-and-out job, we were stuck with no data, no voice, and no way to figure out what the problem might be. Circuit turn-up times came and went, concalls with the switch operators verified that something was on the other end of the loop, but obviously nobody could verify that the NIU talking back to the switch was actually located in the same building, and I certainly didn't see anything on my end. We called the wiring company, and arranged for a 7am meeting the next morning.
Bright and early the next day, we were finally granted access to the wiring closet in the basement. A single shelf was located on a mounting board in the room, containing a few live NIUs and several completely dark NIUs. Of the twelve slots in that shelf, five were marked DEFECTIVE, and only one of the circuit IDs matched an NIU -- the PRI circuit. Not exactly reassuring. We patched that across to the extension, and the PRI circuit lit up. One down, but where's the other circuit?
We proceeded to look in every other wiring closet in the building. All six of them. Nothing. There was another shelf in one closet, but it only had two NIUs, and both were already in use, with foreign circuit IDs. Back to the drawing board. We contacted SBC to pull the original work order. There, in black and white, it said "Shelf 1, slot 5". The shelf in the basement closet had an NIU in slot 5, but it was completely dark. There was nothing else there.
So, we decided to search the room. That's when we discovered the other shelf, located around the corner, behind some boxes. It had a live NIU in slot five. We patched it into the extension, crossed our fingers, and I finally got carrier. By 10am, both circuits were live, and all was finally well.
It's a good thing it all happened by 10am, too, since once the 11am witching hour arrived, that room would be completely inaccessible once more. I can't quite fathom the building management's paranoid behavior regarding that room. In talking with the circuit provider, it took four tries to get that circuit installed in the first place, since the installer had the audacity to show up at noon. They were literally turned away three times before gaining access to the room.
Obviously, this may continue to be a problem. It's hard enough to get line techs to come out and troubleshoot circuit problems, but when the window for gaining physical access to the wiring closet is so limited, it becomes much more difficult. I'll just have to hope that these circuits are trouble-free, or that we can take the wiring closet door off its hinges in an emergency.