I think almost every system we have went down today in some form. We had a motherboard die in a domain controller last night, and Doug and the technician replaced it, but the time for the server was set to PM not AM (it was after one in the morning). Turns out the domain controller was the time server for the network. When we changed it back to AM this morning, everything went haywire. Terminal servers stopped working, the e-mail cluster crashed, accounting systems inaccessible, web portals locked up, backup jobs failed - it was a mess. I got several of the more minor problems resolved, but Doug has been on the phone with tech support for over eight hours trying to resolve the e-mail problem. It's nasty stuff.
LJ just IM'ed me and asked me how the day has been. I told her that it's been terrible. She asked if I had gotten everything fixed, and I said yes, except for the e-mail, and she said, "why isn't that a good day?" I told her that in IT, a good day is when nothing unexpected happens. Sure it's challenging to solve problems, and rewarding to fix a system that lesser mortals would have given up for dead, but most IT folks (at least the ones I know) would rather have things peaceful and boring.
Well, looks like Doug's making progress. Talk to you later.
No comments:
Post a Comment