Information Technologies: File Server Migration (Data Migration)

In my previous post, I talked about choosing and using DFS (Distributed File System) as our new file server design. In this post, I would like to share how we did the data migration. This might not be the best way to do data migration but it works for us. In fact, the process is not complicated but just time consuming.

The data migration was a 3 steps process (restore, synchronise and final cut over) mainly using tape backup and robocopy. The aim is to minimise disruption and make the migration as transparent as possible for the users.

Step 1
If we jump straight to use robocopy, it will take more than a day to complete the copy. So we took the latest full and differential backups of the old file server and restored the data to the new file servers. The restored data on the new file servers will become the base for the robocopy. This will help to reduce the amount of data needed to be copied over the LAN by robocopy.

Step 2
Since the old file server is still up and serving users, the restored data on the new file servers will contain out-dated data over time. Therefore, we schedule robocopy to copy new and changed data from the old file server to the new file servers. We have scheduled the robocopy to run every night for 3 nights. Here a a sample of the robocopy command that we used.

robocopy "\\fileserver\H$\Document\Sales" "\\defile01\H$\Doc \Sales" /E /ZB /copy:DATSOU /XO /r:1 /w:10 /np /log+:c:\robocopy\datacopy.log

\\fileserver\H$\Document\Sales – the path to the old file share
\\defile01\H$\Doc \Sales – the path to the new file share
/E – to include sub-folders, including empty ones.
/ZB - use restartable mode; if access denied use Backup mode. The backup mode is useful is the account you use do not have access right or do not have ownership to the folder/file.
/copy:DATSOU – copy the Data, Attributes, Timestamps, Security ACL, Owner info and Auditing info of the folder/file.
/XO – exclude older file.
/r:1 – retry 1 time for failed copy.
/w:10 – wait for 10 seconds between retry.
/np – don’t show percentage copied.
/log+c:\robocopy\datacopy.log – log the copy to c:\robocopy\datacopy.log. Log file will be appended if already exist.

Since my new file servers are running Windows Server 2008 R2, it comes with robocopy. Otherwise, robocopy is part of Windows Server 2003 Resource Kit Tools.

This step is really to keep the old and new file servers synchronise as mush as possible to reduce the amount of data to copy during the final cut over thus reducing the down time needed. It is also recommended to review the robocopy log file to look out for failed copy and resolve it before the final cut over.

Step 3
We needed to schedule a down time long enough to do a final copy or synchronisation. The estimated time needed to do the final copy or synchronisation can be worked out by reviewing the robocopy log files generated during step 2. Add in a bit of buffer and that will be the down time required. For our case, it is about 2 hours.

Users should not have file server access during this final copy to ensure that the new file servers will have the latest data after the copy. The most appropriate time to do this will be weekend for us and preferably on Sunday. Since we need the old file server to be up and connected to the network; therefore the easiest way to prevent users from accidentally accessing the old file server is to stop the share. For our case, it is to stop the File Share resource using Cluster Administrator.

After we are done with the final copy, logon script needs to be updated to map to the new UNC path (e.g. \\deinfotech.org\Document). Users will have their network drive mapped to the new path at the next logon.

Robocopy is by no mean the only tool available to do the job. I have used XCOPY for adhoc and small file share data copy. There are also other tools such as Secure Copy which has more features but not free.

The next post will be a short one where I will be sharing some of my post file server migration tasks.

If you have other interesting methods to migrate data, I will be really interested to hear from you.