Servers generate quite a bit of heat
Today I bring to you another tale of my Naval adventures in Nuclear IT.
So we have a server space, with 2 cabinets containing 6 servers and a radio communications control panel for those working in the plant to be able to talk to each other. This room has it's own air conditioning, and I remember fondly of being in 100+ degree weather and hopping in this room, of which I and my coworkers had the only 4 keys, to enjoy the 60 degree air. This air conditioner is important, as we shall see...
Neue_Ziel: RIM room, NZ.
Electrician: Hey, are you doing something to the network?
NZ: No, why?
E: Because I can't access my files...
NZ: Okay, let me see....
To my horror, the network is down, and I ping the server that would cause the issue, seeing no reply. I get another call:
NZ: RIM Room, NZ.
Mechanic: (program used for maintenance tracking and safety) is down, and Mechanic 2 says it is doing the same thing for him
A creeping feeling of dread is overcoming me. The user files are on one server and the maintenance program files are on another. I ping the servers, and on a hunch, ping the others in the air conditioned room......no response.
I yell at the one of the guys I work with to come with me and we take off to the server room. Opening the door, we are greeted to temperature alarms and being blasted in the face with very hot air. Using an infrared thermometer, I get 140 degrees on a metal surface.
The air conditioner is off, and resetting the breaker on it does nothing. Using the labelplate on the unit, I am able to locate the feeder panel the A/C unit is on, and find it and all other breakers on it are tagged out. No power, no cool air, causing the servers to shut down on overtemperature conditions at 110F.
I run over to ask what the hell the electricians are doing, and find them taking their sweet time fixing something, and letting the officer in charge of the plant know that their maintenance has taken out the network, and without the network, no one is getting anything done. I see the look of panic in his eyes, and he makes an announcement over the 1MC for the electricians to come. They come in and he says:
Officer: Hurry up and finish your work, NZ says you took out the network.
As mentioned in my other stories, this network is vital for the work, used by lowly third classes up to captains, and no one wants to be the one that brings down the wrath of our department head.
They got power back on, got the A/C on again, and I found that they had failed 2 1 TB hard drives, not bad, but what a pain.
Sometimes user complaints can be helpful.