I got a phone call yesterday afternoon from SDG&E to personally inform me of an unscheduled power outage last night. This was not a computerized recording, it was a real human who identified herself and left the message for me by name. I am impressed by that. SDG&E's really good about sending out form letters in advance of planned outages, but a phone call? Way cool. Anyways, they had to replace a piece of switch gear just outside my subdivision, as a result of something that happened while the city is replacing the sound wall out on Community Road. She said they'd shut us down between 9 and 9:30pm, and have us back on before 11.
Well, it didn't quite work out that way. By 10:15, we still had power, so I walked over there and asked what's up. They were still waiting on the City of Poway to send out traffic control. They finally cut power at 12:45, and restored it about 3am. I still don't have proper UPS monitoring software running on my server, so I just shut it down for the night and let the UPS keep just the DSL modem and the VoIP terminal alive. (Note bene: APC UPSes can be configured to STFU!)
I awoke early this morning, planning to run my car into Sears to have the tires rotated right when they open, and get to work fairly early. I thought this unexpected downtime would be a good opportunity to upgrade the firmware on my 3Ware RAID card. It needed two bouts of that (you can't apply the 6.9 firmware to the 6000 series until you've applied 6.8.) The system booted normally, I rebooted to a DOS floppy, and re-flashed the card successfully. I powered down so the card would reinitialize on powerup. Oops. The system hung after the BIOS detected the IDE chain. Bad. And there went my morning plans.
Three hours later, I've found that the system will boot... if I take out the RAID card. And the mirrored drives have a proprietary partition table on them, so you can't just plug them into the normal IDE ports and have them work. Shit. So I took the card into work, expecting to call 3Ware and arrange an overnight advance replacement. (Crossing my fingers that they still do that with my PCI/PATA Escalade 6400.) Just for the hell of it, I threw it into a machine at work. It came up fine.
Well that explains it. The slot on the home server is now bad. The motherboards on my PCs are falling victim to the counterfeit capacitor problem, and have started to bulge and leak already. Both of these machines' days are numbered. I got home, popped the RAID back into a different slot, and it booted just fine.
Not that it's all bad... while I had it open, I spotted two bad case fans. The LED fans at Fry's were selling for the same price as their black ones, so I figured WTF and bought a couple green ones. Why green? Because green is not-purple, and this machine is named babylon5. I cleaned up the cable mess a bit too, pulled the old SCSI tape drive I don't use and the DVD drive that's gone bad, and screwed some guards over the fans so they won't jam on cables anymore. And here we are back in business...