Friday, October 1, 2004

My thirteen days in Exchange Hell


By David Gewirtz
All hope abandon, ye who enter in. -- Dante

You can tell the difference between Mac, Linux, and Microsoft experts by how they react to you when you've got a systems problem. To the Mac expert, you're simply not stylish enough to get help. To the Linux user, you're simply not smart enough to fix it yourself. But, if you approach a Microsoft systems expert about a server problem, you get a completely different reaction. The Microsoft expert will look at you with compassion, reach out gently, and say, "I share your pain."

Back in March, I had a lot of Microsoft experts sharing my pain. Our Exchange server crashed.

In the story of Dante's Inferno, Dante finds himself lost in a dark wood and threatened by wild animals that block his path. The ghost of the poet Virgil appears to him and tells him the only way out is through Hell itself. Reluctantly, Dante agrees to make this journey.

"For the next thirteen days, I attempted to restore the Exchange backup."

This nearly perfectly describes my thirteen days attempting to get our mail stores back. At nearly every turn, Exchange blocked my path, refusing to let me back in. Eventually, I had to live through and learn far more about Exchange than I ever wanted, and had to travel through many, many levels of Exchange Hell, before I finally found my way out.

Let me give you some background. A few years ago, when we started OutlookPower Magazine, we decided we need to be running Exchange 2000 as our mail server. This would give us a much better idea of the environment, and give us material to write about.

Setup was straightforward. We got ourselves up and running in a short time, and have been running smoothly ever since.

We were very well-behaved Exchange users. Every night, we ran a backup using Windows 2000's built-in backup program. Every week we rotated the backups. And monthly, we swapped the backups off-site.

Life with Exchange was good. Oh, one day, about a year ago, for no descernable reason, we lost all our public folders, never to be found again. But generally, Exchange was solid and reliable.

We built our Exchange server on a tough server box. We had a monster power supply, we mirrored our data drives, and kept up with OS upgrades.

Like I said, we were quite well-behaved and we had a nice, solid installation.

In March, it all went to Hell.

The beginning of the end

If you listened carefully to the Exchange server, you could hear a sound that just wasn't right. We started to hear a grinding sound coming from the Exchange server. It wasn't any of the fans; it turned out to be the C drive, getting ready to fail.