Wednesday, September 9, 2009

Disaster Recovery for a failed Exchange 2000 Server

Recently, did a disaster recovery for a failed Exchange 2000 Server. Onsite for about 12 hours at the start of a long weekend...here's a snapshot of what was done.
Links: How to move Exchange 2000 to new hardware and keep the same server name ; How to remove data in Active Directory after an unsuccessful domain controller demotion; XADM: How to Reset System Folders on an Exchange 2000 Server
Situation: Hardware running Exchange 2000 Standard (Exchange) and Domain Controller/Global Catalog (DC/GC) roles has failed. The failed server was also a file share and running GFI Faxmaker. A rebuild on new hardware was required. FSMO roles on domain controller were already seized. Blackberry Enterprise Server (BES) is running on a separate server. There was a System State backup and a brick-level backup of the mailboxes. There were no full backups of the mailbox database or public folder store.
Resolution:
1. Obtain hardware.
Obtain adequate hardware to set up Exchange and DC/GC on. The hardware obtained at short notice seemed to not have a Windows 2000 network driver and no other suitable network card was found. The installation of Windows Server 2000 could not be completed.
2. Installed Windows Server 2003 R2 and installed Service Pack 2 and other windows updates. This server was standalone and not part of the domain.
3. Installed and configured Virtual Server 2005 R2.
4. Windows Server 2000 Standard Virtual Machine was set up. It was configured with the same name and IP address as the failed Exchange server. It was not joined to the domain. Two virtual disks were created – i.e. C: drive 15GB and D: 127GB. The disk drive letters matched those on the failed server.
5. Installed backup software (Backup Exec) on Windows Server 2000 virtual machine.
6. Removed failed DC from Active Directory.
Refer to KB216498. Steps under Procedure 2: Windows 2000 (All versions) Windows Server 2003 RTM were followed. ADSIEdit was used to delete the computer account. The UserAccountControl attribute needed to be set to 4096 before it could be deleted within ADSIEdit. Active Directory Sites and Services was also used to remove the failed domain controller.
7. Joined the Windows Server 2000 virtual machine to the domain.
8. Added components required for Exchange 2000 – eg. IIS (with ASP.NET, SMTP, NNTP).
SMTP service was set to disabled and stopped.
9. Turned off SMTP inbound on firewall.
10. Promoted virtual machine to a domain controller (as an additional domain controller in an existing domain).
11. Rebooted virtual machine and set as a Global Catalog.
12. Changed account to start Backup Exec services to Domain\Administrator as these credentials changed with the promotion to a domain controller.
13. Rebooted virtual machine.

14. Attempted to run setup /disasterrecovery to install Exchange. Kept getting the following message when trying to run setup /disasterrecovery switch to reinstall Exchange 2000:
The component "Microsoft Exchange Messaging and Collaboration Services" cannot be assigned the action "Disaster Recovery" because: - Active Directory has not replicated all the correct permissions for the deleted items container. Please wait until replication completes before running setup.
15. Reran setup /forestprep and setup /domainprep.
16. Ran setup /disasterrecovery successfully to install Exchange 2000 in disaster recovery mode.
17. Ran upgrade /disasterrecovery to install Exchange 2000 SP3 in disaster recovery mode.

18. Set ServicePackBuild key. The following registry key did not exist so it was created:
HKEY_LOCAL_MACHINE\Software\Microsoft\Exchange\Setup
DWORD value name: ServicePackBuild
Hexadecimal value: 1869 (for Service Pack 3)
19. Installed any hotfixes that were applied to failed Exchange server. Note if the ServicePackBuild key was not done correctly, hotfixes may fail.
20. Set SMTP service to Automatic and started the service.
21. Started mailbox store. This will start up as a dialtone (empty) database. Mailboxes will begin to appear in System Manager when there is a change to their mailbox (i.e. send/receive mail).
22. Took mailbox store out of disaster recovery mode. In Exchange System Manager, right-clicked the mailbox store, and then clicked Properties. On the Database tab, cleared the Do not mount this store at start-up check box. Clicked OK to save the setting.
23. Started public folder store. This will start up as a dialtone (empty) database.
24. Took public folder store out of disaster recovery mode. In Exchange System Manager, right-clicked the public folder store, and then clicked Properties. On the Database tab, cleared the Do not mount this store at start-up check box. Clicked OK to save the setting.
25. Blackberry Enterprise Server issues. The BES was running on another server. Restarting the services didn’t seem to work and all blackberries were in initializing mode and not receiving mail. An encryption error message appeared on the blackberry. Removed one blackberry and readded it and set up a password for enterprise activation. After activation was complete, the blackberry started working. A wipe and reload was not necessary.
26. Reset System Folders. Since there was no backup of the public folder store, free/busy folders and any public folders were lost. Followed KB275171 to reset system folders in Exchange 2000.
27. Re-enabled SMTP inbound rule on Firewall. There was an issue with redirected folders to a share on failed server that caused an issue trying to save the changes and re-enable this rule. Once the folder and share was re-created, the rule was re-enabled without issue.
28. Performed restores of mailbox data from bricklevel backups. This worked pretty slick but was time-consuming due to space restrictions and performing restores one mailbox at a time.
29. Faxmaker install and configuration. This will need to wait until permanent hardware is available that will accept fax card.

2 comments:

Jhon Drake said...

Great article, I tested this exchange recovery solution from http://www.lepide.com/exchange-manager/ which helps to recover corrupt or inaccessible exchange database or mailboxes from exchange database and restore to new outlook pst files. This tool to recover corrupt edb files data and convert to pst files. It allows to extract exchange server backup from the backups created using VERITAS, NTBackup, Symantec.

smithjake said...

To try this exchange server recovery tool to recover corrupted exchange server database and restore exchange data to MS Outlook PST files data. Exchange Server Recovery Software fixes exchange error and export multiple mailboxes into Outlook PST files and recover hard deleted items from database and restore all mailboxes and public folders in structured format by saving the original data properties. https://softcart.wordpress.com/exchange-mailbox-recovery/